Methods for Searching for Best Digital Color Options for Reproduction of Image-Based Layouts Created through an Electronic Interface

ABSTRACT

A system for creating and editing image-based projects includes a server connected to a network, the server having access to least one processor and a data repository, the server including a non-transitory physical medium, and software running from the non-transitory physical medium, the software providing for establishing a client server connection between the server and at least one user-operated computing appliance connected to the network, initiating and maintaining an active data session between one or more users involved in project creation and or in project editing through a graphics user interface (GUI);, analyzing in at least three dimensional values, all of the colors of an asset presented in the image-based project, creating a histogram of the asset based on the analysis by the third function, blending the histogram by adding new color contributions to augment the original colors in relevancy, and storing the blended histogram with the asset.

CROSS-REFERENCE TO RELATED DOCUMENTS

The present invention claims priority to a U.S. provisional patentapplication 61/707,007, filed Sep. 28, 2012 and entitled “Method andApparatus for Creating Photobooks”, disclosure of which is incorporatedherein in its entirety at least by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention is in the field of electronic commerce andpertains particularly to methods and apparatus for streamlining creationand editing of photo-based projects using a graphics user interface on adata network.

2. Discussion of the State of the Art

In the field of electronic commerce also known as ecommerce, there areproject-oriented Websites where users may create and edit worksincluding models, poems, art, and photo-based products. Websitescatering to photo-based editing promote online publishing and onlinetransacting relative to printed works. The current state of suchelectronic services leaves much to be desired. Many of these serviceshave fixed templates and multiple interfaces for project editing, makingit tedious to get the desired output.

Other problems with the current state of photo-based services includeinflexibility of the editor interface relative to process steps forediting content. Most such processes are not transparent and could leadto errors in printing and therefore in the final output product.Furthermore, each typical project is initiated from scratch in a waythat does not promote efficiency from start to finish of a project.

The inventors are aware of a graphics user interface (GUI) for creatingand editing photo-based projects over a data network that providescreation and editing tools to users through a single interface. Theinterface and supporting back-end system provides some automated andsemi-automated processes, which users may take advantage of to enhancetheir experience with the mechanics of creating and editing animage-based product such as a photo book for example.

It has occurred to the inventor that there are still many opportunitiesto improve upon existing system functionality and to expand the reach ofthe system to provide additional functionalities not only relevant tothe aspects of creating and editing projects through the interface, butalso relevant to other aspects of a user's experience and of systemback-end operations including network activities undertaken to fulfillclient orders of created projects.

Therefore, what is needed are new methods for further streamliningcreation, editing, and printing of photo-based project using a graphicsuser interface over a digital network.

SUMMARY OF THE INVENTION

In one embodiment of the invention a system for creating and editingimage-based projects is provided, comprising a server connected to anetwork, the server having access to least one processor and a datarepository, the server including a non-transitory physical medium, andsoftware running from the non-transitory physical medium, the softwareproviding a first function for establishing a client server connectionbetween the server and at least one user-operated computing applianceconnected to the network, a second function for initiating andmaintaining an active data session between one or more users involved inproject creation and or in project editing through a graphics userinterface (GUI), a third function for analyzing in at least threedimensional values, all of the colors of an asset presented in theimage-based project, a fourth function for creating a histogram of theasset based on the analysis by the third function, a fifth function forblending the histogram by adding new color contributions to augment theoriginal colors in relevancy, and a sixth function for storing theblended histogram with the asset.

In one embodiment the network is the Internet network. Also in oneembodiment a color palette is defined from the original analysis of thethree dimensional color values. Still in one embodiment tag terms andcolor terms are used to search for assets that have been analyzed andstored for use. In another embodiment contributions added to the colorpalette for an asset are added as a result of a color blendingoperation.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

FIG. 1 is an architectural overview of a communications networksupporting network-based graphics manipulation and production of imageand or text-based products according to embodiments of the presentinvention.

FIG. 2 is an exemplary screen shot of an introduction page of Website ofFIG. 1 according to an embodiment of the present invention.

FIG. 3 is an exemplary screen shot of a login/sign-up page presented toa user as a result of interaction with the introduction page.

FIG. 4 is an exemplary screen shot of a welcome page presented to a useras a result of successful login to the site.

FIG. 5 is an exemplary screen shot of a project configuration pagepresented to the user as a result of interaction with the Create optionof the page of FIG. 4.

FIG. 6 is an exemplary screenshot of size-selection page for selectingan initial orientation and size for a project.

FIG. 7 is an exemplary screen shot of a theme-selection page forselecting an initial theme for a project.

FIG. 8 is an exemplary screen shot of a Mixbook Editor™ (MBE) userinterface (UI) for editing a project in progress according to anembodiment of the present invention.

FIG. 9 is an exemplary screen shot of an add-photo page for acquiringphotos to add to a project according to an embodiment of the presentinvention.

FIG. 10 is an elevation view of a photo spread of a photo-book projectexhibiting a dynamically editable text box according to an embodiment ofthe present invention.

FIG. 11 is an elevation view of the photo spread of FIG. 10 with thetext box moved to a position of conflict relative to a print parameterapplied to a printing process for the photo-book.

FIG. 12 is an elevation view of a photo spread of a project illustratingreal-time text color application using theme colors.

FIG. 13 is an elevation view of photo spread of the same projectillustrating real-time font color change using system colors.

FIG. 14 is an elevation view of a photo spread illustrating aphoto-editing frame and intuitive photo panning interface according toan embodiment of the present invention.

FIG. 15 is an elevation view of the photo spread of FIG. 14 with a styleediting interface 1501 invoked from photo editing toolbar according toan embodiment of the present invention.

FIG. 16 is an exemplary screen shot of a contributor invitationinterface according to an embodiment of the present invention.

FIG. 17 is an exemplary screenshot of a proxy-login interface thatenables a user or contributor to gain access to a personal account tolink to or otherwise acquire access to photos available through thepersonal account.

FIG. 18 is an exemplary screen shot of an add friends interface foradding friends to the service of the present invention.

FIG. 19 is an exemplary screen shot of a preview interface forpreviewing a finished project according to an embodiment of the presentinvention.

FIG. 20 is an exploded screenshot of the background selection portion ofthe user interface of FIG. 8 including an advanced background touch-upsection according to an embodiment of the present invention.

FIG. 21 is a page-on-canvas view illustrating a relative page coordinatesystem according to an embodiment of the present invention.

FIG. 22 is a block diagram illustrating a grouping of portrait pagesizes exhibiting similar aspect ratios.

FIG. 23 is a process flow chart illustrating steps for displayingproject page content at a core aspect ratio and optimizing the contentfor printing at a specified size.

FIG. 24 is a process flow chart illustrating steps for migrating contentfrom one aspect ratio grouping to another aspect ratio grouping.

FIG. 25 is a block diagram illustrating system architecture forinterpolating sample and default data into a project template using aspecial markup language according to an embodiment of the presentinvention.

FIG. 26 is a front view of a photo frame that can be automatically sizedto fit a photo according to an embodiment of the present invention.

FIG. 27 is a front view of photo frame automatically-sized about aphoto.

FIG. 28 is an exemplary screenshot of an editing interface displaying aphoto spread across multiple pages (front cover, back cover, and spine)according to an embodiment of the present invention.

FIG. 29 is a block diagram illustrating an asset recommendation systemaccording to an embodiment of the present invention.

FIG. 30 is an architectural diagram illustrating a printing networkaccording to an embodiment of the present invention.

FIG. 31 is a process flow chart illustrating steps for printing largeoutput images according to an embodiment of the present invention.

FIG. 32 is an elevation view of a photo book project undergoing editingfor dimensional printing according to an embodiment of the presentinvention.

FIG. 33 is a block diagram depicting an incentive determination anddelivery network according to an embodiment of the present invention.

FIG. 34 is a screen shot of a photo book project that is open anddisplaying two pages, generally referred to herein as canvas.

FIG. 35 is a process flow chart depicting steps for configuring itemsfor interactivity according to an embodiment of the present invention.

FIG. 36 is an architectural view of a service network 3600 supportingmanagement of print order distribution according to an embodiment of thepresent invention.

FIG. 37 is a process flow chart depicting steps for routing a printorder based on analysis of data.

FIG. 38 is an architectural overview of a customer project creation andfulfillment network supporting default and dynamic styling of photoscontributed to image and or text based projects.

FIG. 39 is a block diagram depicting a fulfillment network supportingautomated font style and size application during remixing of a pagelayout.

FIG. 40 is a process flow chart depicting steps for preserving layerdepth for an item moved between pages in a layout.

FIG. 41 is a process flow chart depicting steps for preparing graphicand text assets in a project for printing.

FIG. 42 is a process flow chart depicting steps for optimizing storagespace and data reproduction performance relative to an image ortext-based layout with content.

FIG. 43 is a project flow chart depicting steps for optimizingperformance in content delivery using a template with different datalayers.

FIG. 44 is an architectural overview of a network supporting enhancedcolor options for displayed content.

FIG. 45 is a process flow chart depicting steps for applying motiondynamics for digital assets.

FIG. 46 is an architectural overview of a fulfillment network thatsupports automatic recognition of and captioning of photos contributedby clients building or editing an image and or text-based projectthrough a graphics user interface (GUI).

FIG. 47 is a process flow chart depicting steps for matching facialrecognition patterns in photos to pre-identify photo contributions to aproject.

FIG. 48 is an elevation view of a double page canvas layout supportingdynamic control over gridline snap-to functionality.

FIG. 49 is a process flow chart depicting steps for simulating displayof an uploading photo according to an aspect of the present invention.

FIG. 50 is a process flow chart depicting steps for simulating higherphoto resolution on a display screen for user visualization.

FIG. 51 is an architectural diagram of a network supporting tracking oftheme-based assets according to an embodiment of the present invention.

FIG. 52 is an entity relationship model depicting relationships betweentags, tag groups, templates and template groups that share themesaccording to an embodiment of the present invention.

FIG. 53 is a process flow chart depicting steps for offering an upsellto a client involved in a transaction.

FIG. 54 is a process flow chart depicting steps for dynamic bundling anddelivery of editing tools in an editor based on user preference andproject analysis.

FIG. 55 is a process flow chart depicting steps for marking data forpersistence according to an embodiment of the present invention.

FIG. 56 is a process flow chart depicting steps for persisting data at asubsequent user session with the same project.

FIG. 57 is an architectural overview of a network supporting contentproposals based on information gathering according to an embodiment ofthe present invention.

FIG. 58 is an exemplary screen shot of a graphics user interfacesupporting photo selection and management for project enhancementaccording to an embodiment of the present invention.

FIG. 59 is a process flow chart depicting steps for generating a mosaicpattern of photos for an image-based project according to an embodimentof the present invention.

FIG. 60 is an architectural overview of a fulfillment network supportingthe initiation, building, and editing of image and or text-basedprojects from a wireless computing appliance.

FIG. 61 is a process flow chart depicting steps for form fieldpopulation on a space-constrained device according to an embodiment ofthe present invention.

FIG. 62 is a process flow chart depicting steps for auto-populatingaddress information for clients filling out forms according to anembodiment of the present invention.

FIG. 63 is a block diagram illustrating a mobile appliance supportingpreview of an order for an image and or text-based product created froma mobile application according to an embodiment of the presentinvention.

FIG. 64 is an architectural overview of a fulfillment network supportingphoto contribution to image-based projects created through a graphicsuser interface.

FIG. 65 is an architectural overview of a fulfillment network supportingprovision of colorable assets to users engaged in building and editingimage-based projects through a graphics user interface (GUI).

DETAILED DESCRIPTION

The inventors provide a unique network-based graphics-manipulationsystem along with a set of novel manual and automated methods andprocesses that enable users operating network-capable computingappliances to efficiently create a variety of image and or text-basedproducts with or without collaborative effort. The systems and methodsof the present invention are described in enabling detail using thefollowing examples, which may represent more than one embodiment of theinvention.

System Architecture

FIG. 1 is an architectural overview of a communications network 100supporting network-based graphics manipulation and production of imageand or text-based products according to embodiments of the presentinvention. Communications network 100 includes a wide-area-network (WAN)that is in most embodiments the well-known Internet network. WAN 101 maybe a private, corporate, or publicly-accessible data network withoutdeparting from the spirit and scope of the present invention. WAN 101may be referred to throughout this specification as the Internet 101.Internet 101 is further illustrated by way of a network backbone 116,which represents all of the lines, equipment, and access points thatmake up the World Wide Web (WWW) as a whole including any connectedsub-networks. Therefore, there are no geographic limits to practice ofthe present invention.

Internet backbone 116 supports a Web server (WS) 105. It is noted hereinthat all network-capable computing appliances such as physicalcomputers, servers, and other computing machines discussed in thisspecification shall be assumed to contain, be coupled to, or otherwisehave accessible thereto, one or more digital mediums adapted to containthe data and software required to support and enable the primaryfunction or functions (in the case of multi-task machines) of thecomputing appliance.

WS 105 is an electronic information server connected to Internet 101 andadapted to server information pages or Web pages as they are known inthe art. WS 105 may be maintained by a third party that provides aWebsite hosting service. In one embodiment a service provider providingthe service of the present invention owns and maintains WS 105. WS 105hosts a Website 106, through which, users may access elements of thepresent invention. Website 106 is adapted to provide authenticatedaccess to users who create, publish, print, and purchase image and ortext-based products such as photo-books, photo-cards, photo-calendars,photo-posters, and related products.

Because all of the products are image and or text-based and createdusing a graphics manipulation system provided by the inventors as anetwork-based software application, the relative term descriptive of theoverall process and for the providing company is Mixbook™. The termMixbook™ may be used throughout this specification to describe theoverall image and or text-based product creation and editing process ofthe invention, the Website through which the process is made accessibleto authorized users, and the service-providing company.

Communications network 100 also includes an access network (AN) 104,which may represent any data network adapted to provide access toInternet network 101. AN 104 may be a public-switched-telephone-network(PSTN) or some other public or private telephony network. AN 104 may bea local wireless network, a cellular time division multiple access(CDMA) network, a Wireless Fidelity (WiFi) network, or any othercandidate access network through which one may access Internet 101.

User may access WS 105, more particularly Mixbook™ Website 106, throughan Internet access facility 107 in access network (AN) 104 and anInternet access line 120. Internet access facility 107 may be maintainedand operated by an Internet service provider (ISP) or by a wirelessInternet service provider (WISP), which ever is appropriate for anyparticular Internet connection. Users who might access WS 105 arerepresented herein as network-capable computing appliances, moreparticularly, a laptop computer 122, a desktop computer 123, and a smarttelephone 124. Each mentioned appliance may be assumed to beInternet-capable by way of one or more network browsing applicationsresiding thereon and executable there from.

Desktop computer 123 is connected to an Internet-connection server 107by way of an Internet access line 126. Desktop computer 123, onceconnected, may access Website 106 hosted on WS 105. Desktop computer 123has one or more input devices (not illustrated) coupled thereto such asa keyboard, a mouse, and a microphone for (Speech-to-Text Commands).Smart phone 124 may connect wirelessly via a wireless link 127 to anInternet service provider (also represented by machine 107) thatestablishes a wireless Internet connection like public WiFi for example.Smart phone 124 may or may not include a keyboard input device. In oneembodiment smartphone 124 has a touch-screen display. Laptop 122 isillustrated as connected wirelessly to Internet 101 via WISP 107 usingwireless link 128. Laptop 122 includes a keyboard and/or other inputdevices as may be appropriate.

Laptop 122, desktop 123, and smartphone 124 each include an Internetbrowser application (not illustrated) for accessing and navigatingnetwork 101. Backbone 116 supports a runtime server (RS) 109 adapted tohost a Mixbook™ user-interface 110. User interface 110 is accessible toall Website visitors for the purpose of creating, manipulating, andprinting image and or text-based products such as a photo collage book,for example. Users accessing Website 106 who are ready to create aproduct may be re-directed to RS 109.

Each user authorized to create and publish an image and or text-basedproduct using the site may receive a small download containing a compactediting package known as the Mixbook™ editor (MBE) to the inventors. MBE125 a is installed as a browser-based extension or plug-in in oneembodiment on desktop computer 123. Laptop 122 has an instance of MBE125 b installed as a browser-based extension or plug-in. Smartphone 124has an instance of MBE 125 c installed as a browser-based extension orplug-in. An instance of MBE may be customized for any computingappliance that may access the Internet and through which a user may seeand edit content. Therefore, MBE instances 125 (a-c), though the samebasic extension or plug-in, may contain differences based on hostrequirements. In one embodiment of the present invention there are nosoftware downloads required in order to practice the present invention.In this case the Mixbook™ editing SW may be server hosted only. Inanother embodiment, the Mixbook™ editing SW may be ported to a desktopapplication such as Adobe Air™ and thus be operated as a desktopapplication. In one embodiment the SW is included as an add-on featureto any suitable desktop application and may be installed on a computinghost with that desktop application from a removable medium such as a CDROM, for example.

Service provider facility 107 includes a connected data repository 108labeled Data. Data repository 108 contains all of the customer contactand billing information for the Internet service provider. One withskill in the art will appreciate the many possible internet connectionschemes. It is preferred in most embodiments that user have a high speedInternet connection for the purpose of manipulating and editinggraphics, which can be bandwidth intensive. The inventors provide one ormore innovative solutions for saving bandwidth while editing images andimage products online making the entire experience more efficient andeasier for users practicing the invention.

The Mixbook™ Website 106 establishes a community-based portal and socialinteraction site that revolves around creating, editing, sharingpublishing, printing, and purchasing image and or text-based productscreated online by one or more user working in collaboration together.Users such as those operating appliances 122-124 connect online andnavigate to WS 105 to access Website 106. When any user determines tocreate an image and or text-based product like a photo album, forexample, they are directed to an appropriate portal server like RS 109hosting Mixbook™ user interface (UI) 110. UI 110 is adapted to provideall of the assets needed to create and publish complete image and ortext-based products. Image and or text-based products created throughWebsite 106 include products containing images uploaded by one or moreauthorized users. Any work in progress or completed is termed a project.A project may be initiated by one user whereby other users are theninvited by the initiating to join the creative experience incollaboration and those users may contribute their own photos to theproject. Such a project may be published for viewing by all or some ofthe community. Finished projects may be printed and distributed as “hardproducts” available for purchase by members of the community.

In one embodiment persistent storage of graphics uploaded by communitymembers to be included into projects is obtained (leased) through athird-party storage provider. In this example a simple storage service(s3) data storage cloud 102 is illustrated and made available for use bya third-party service provider such as Amazon™. A storage server 112 isillustrated within s3 cloud 102 and has connection to Internet backbone116. SS 112 may be one of many servers including associated mass datarepositories connected to SS 112 such as repository 113 contained withinstorage cloud 102. In this logical representation all of the graphics(photos) that are uploaded to insert into projects are stored in arepository such as repository 113 in storage cloud 102. Repository 113may be an optical, magnetic, or some other type of data storagefacility. In one embodiment the mass computations required for real-timeand transparent editing and collaborating on multiple projects areperformed by virtual machine instances 120 in a computing cloud 103. Inanother embodiment the service host may maintain one or more powerfulcomputing machines and storage devices for performing computations andfor storing graphics for users of the service.

In use of the present invention a user such as one operating one ofcomputing appliances 122-124 connects online and accesses Mixbook™Website 106 and logs into the site. If the user has already registeredand created a project, a Web page personalized to that user is servedthat includes all of that user's projects in the latest states ofprogress. The user may enter any project for which she or he hasauthorization to contribute to and may review, edit, or otherwise workthose projects. Photos uploaded by the user to include into projects maybe stored in cloud 102 and served to the projects when needed by thesystem. Any computing such as editing resizing, alterations, and so onmay be handled in cloud 103. A user may, through Website 106, authorizeother users registered with the service of the invention to collaborateon a project initiated by that user. In one embodiment a user having aproject initiated may invite other potential users to the site so theymay, with permissions, enter the site and collaborate with that user.

In one embodiment of the present invention photos that are stored on theInternet can be represented in any online project provided the user hasauthorized access to those photos. For example, a user who has one ormore accounts to third-party social interaction networks like MySpace™,Facebook™, Photobucket™, Flickr™, or similar sites may use photos ofregistered friends that they are allowed to access in their own personalprojects. These photos do not have to be uploaded to the service of thepresent invention. Rather, these photos can be accessed from theirpresent storage locations anywhere on the Internet provided that thestorage system is online.

Internet backbone 116 supports a Facebook™ server (FB) 114 coupled to adata repository 115 for storing images and other graphics. Internetbackbone 116 supports a Myspace™ server (MS) 111 coupled to a datarepository 117 adapted to store images and other graphics. Backbone 116supports a Flickr™ server (FL) 118 coupled to a data repository 119adapted to store images and other graphics. Any of these images can beserved to an active project by the respective servers directly fromtheir storage locations. Any of these images that are altered duringproject work or collaboration may be subsequently saved to third-partys3 storage cloud 102 in repository 113 controlled by SS 112.

In one embodiment of the present invention Mixbook™ Website 106 includesor has working access to a SW print engine (not illustrated here) thatis adapted to render any Mixbook™ project for professional printing. Inone embodiment printing is performed by a third-party provider who mayalso ship or distribute finished products for a price. In one embodimenta user may access the service through Website 106 and may initiate andcomplete a project that will be printed for that user for a static pricefor that particular product.

It is noted herein that a project created on Mixbook™ may be efficientlygathered for print in virtually any print format Likewise, contentoriginally laid out in one format or aspect ratio may be efficientlyconverted to another layout before printing in an automated process thatincorporates the typical size and layout changes necessary to convertfrom one layout to another automatically. Furthermore, content assembledfor a particular layout and product type may be automatically convertedfor print output in another layout and product type. These and otherunique advantages over current systems will be made apparent in thefollowing examples.

FIG. 2 is an exemplary screen shot of an introduction page 200 ofWebsite 106 of FIG. 1 according to an embodiment of the presentinvention. Introduction page 200 includes interactive indicia 201, whichincludes an option for retrieving help, signing up for service, and forlogging into the service. On the same title bar containing theinteractive indicia 201 for signing up or signing into the process,there are the navigation options Learn More; Pricing; and Gallery. Theoption “learn more” leads to additional information about the servicesprovided, such as product quality, use cases, etc. The pricing optionbrings up one or more complete pricing structures for the availableproducts that can be created and printed. The option Gallery is anoption that provides access to completed projects authorized for publicviewing.

Page 200 contains more than one feature advertisement 202. Each featureadvertisement may comment or provide some input, access to, orinformation about a feature of the service or fact about the providingcompany. In the first feature advertisement (center top) there is aninteractive option 204 for getting started now. Invocation of thisinteractive link may launch a registration or sign-up process followedby an opportunity to begin a Mixbook™ project. Hyperlinked informationis provided on introduction page 200 under categories such as LearnMore, Our Services, Idea Center, Programs, and Corporate.

In one embodiment, page 200 includes an option “try it now”, which maybe replace option 204. Invoking such an option may take the “guest” to aMixbook Editor™ without requiring them to login or have an account withthe site. Transparent to the user, the system creates a temporaryaccount for them. If the new guest user attempts to save any work on thesite they are asked to register as a user.

FIG. 3 is an exemplary screen shot of a login/sign-up page 300 presentedto a user as a result of interaction with the introduction page.Login/sign-up page 300 includes a log-in interface 301 for entering anemail address and password for authenticating the user to use the site.A login button 303 is provided for submitting the authentication dataand logging onto the site. If a user has reached page 300 and has notyet signed up for the service of the present invention, the user may bepresented with various benefit statements and a signup now button, whichwhen invoked calls up a registration interface.

FIG. 4 is an exemplary screen shot of a welcome page 400 presented to auser as a result of successful login to the site. Page 400 is a welcomehome page personalized for the user. Page 400 includes navigationoptions 402 including the locations Home, Create, Pricing, and Gallery.The link Pricing takes a user to a pricing page detailing differentprices for ordered products. The Link Gallery take a user to a galleryof public Mixbook™ products created by community members who have givenpermission to view their created products.

Page 400 includes a personal welcome and may also include information405 including status indication of an email account provided to the useras part of the service. Information 405 also includes accountinformation, a Help option, an option for logging out, and a link to ashopping cart for purchasing Mixbook™ products. A user may purchaseproducts created by that user or products created by others that theuser is authorized to purchase. Page 400 includes a sidebar area 403that includes several navigation links such as My Home, My Books, MyFriends, My Photos (uploaded) and My Inbox (provided mail account).

Page 400 includes a workspace window 404 that contains an interface 406for uploading a picture for creating a profile. Interface 406 includes adata entry field to enter the path of a picture or the user may browsethe commuting appliance for a profile picture. An upload button isprovided to effect upload of a profile picture. A place holder 409 isprovided to display the uploaded picture. After creating a profile anduploading a profile picture, the profile picture appears as a profilelink 408. Page 400 has a site search button 411 adapted to enable thatuser to search the Website using a keyword or phrase.

Page 400 includes a status report or listing 407 that tells the userwhat projects are active or available to the user. This may includeprojects created by the user and any projects initiated by others thatthe user has authorized access to. In this example, the user has yet tocreate a Mixbook™ product. Listing 407 includes an interactive commandoption (Create One) for creating a product such as a photo-book forexample.

FIG. 5 is an exemplary screen shot of a project configuration page 500presented to the user as a result of interaction with the Create optionof the page of FIG. 4. Page 500 includes a workspace window 501 forentering information about a project that a user wants to initiate.Information fields 504 include a project title field, a projectdescription field, a field for entering tags for the project, which maybe used later by friends of the user or by other community members tosearch for the project. Fields 504 also include a category field forselecting a category of a project.

A configuration option 502 is provided for electing who may view theproposed project. In this example the project will be viewable byfriends of the user. An invitation window 503 is provided for enablingthe user to invite or add contributors to the project. A contributor isany of the user's friends or any other community member that the userinvites to contribute. In one embodiment a project may be initiated by auser and may be designated open to all contributors or all of the site'scommunity members. Online community members represent an active networkestablished to aid book creation.

The inventors emphasize the idea of a community that develops aroundcreating projects such as photo book. More importantly the systemstreamlines the act of starting projects by creating a socialinteraction network on the site to help propagate book creation. Forexample, by linking with friends and family on the site, the user canutilize the platform for inviting other site users to collaborate on aproject such as a photo book with them, rather than having to entertheir email addresses. Members of this network are community memberstermed “Friends” from the perspective of an inviting user.

When the user begins a project such as a photo book and enters bookinformation, the user is enabled to select members from his network(friends) already registered with the site to help work on it. Thisallows the user to quickly select the appropriate members to contributematerial to the book without having to enter their email addresses.

The system of the present invention enables real-time collaboration.Users may interact with one another in real time as they collaborate ona project. A user may be a contributor without necessarily collaboratingin real time with the creator or other users on a project. Thetransparency of collaboration is such that users may communicate witheach other while working on a particular book and may also viewreal-time editing and creation of that book while it is happening.

As multiple contributors work on a multi-page project such as aphoto-book for example, they are able to view the edits made by otherusers in real time such as adding, changing, and removing pictures andtext. For example, a user that is viewing a page that a contributor isworking on will see a picture move or some other changes happening onthe page in real time.

The system supports presence reporting so any user can see real-timestatus of all the other current contributors or editors of the projectand may invite any of them to collaborate on the page(s) the user isworking on. When two users are on a same page, they can annotate andchange the page in real time, and see each other's mouse cursors in realtime as they move across the screen. When a user selects friends and/orfamily to contribute to a project such as a photo book at the timeproject initiation or by selecting the collaboration feature after thebook as been started, those selected friends or family will be sentautomated invitations to contribute to the selected book. Once a friendor family member accepts such an invitation they will automatically beapproved as become a contributor of the book.

Members of the network who are contributors to a project will be able toadd various materials to the project including but not limited tophotos, videos, text, and pages that have already been created on thesite. When a new project is created the creator and/or contributors ofthe new project such as a photo book may take pages from other existingphoto books on the site and may use them in the new book. In this way,many different projects may use the same project pages. When membersbecome contributors to a book they have the option of adding any pagesthat they have previously created on the site in addition to pages oftheir friends that they have permission to use and any publiclyavailable pages on the site. Adding pages from other projects can bedone two different ways.

In one embodiment a same page added to a project that exists in one ormore other projects may be referenced so that any changes made to thatpage affect every project that has this page present. In anotherembodiment a same page found in another project and contributed to thenew project is copied so that any changes made to the copy will notpropagate to the original page. Members of the site may search publicprojects and pages and may use any public pages they find on the site. Apublic page is a page created by a member of the site that is allowed tobe used by any member in any book. Users will have the option to usethese pages in any book that they create or contribute to. In oneembodiment part of a project may be public while other parts of theproject are held private. A public page may be embedded in a projectthat has pages private pages not made public.

A server such as RS 109 of FIG. 1 facilitates transfer of informationbetween users collaborating in real time on a project. For example, iffive collaborators are in the process of editing a book, the server canuse open socket connections to read when each user is making a changeand then push the change data in an unobtrusive way to the othercollaborators. The Mixbook™ editor (MBE) enables live messaging betweenusers or friends that are currently collaborating on a project wherebyall of the users are using the MBE. In this way users may exchangeideas, tips and other information as they edit a project together.

Invitation window 503 may be expanded to enable a user to view any ofthe user's friends that are registered with the service as “friends” ofthat user. In one embodiment those friends are flagged as being “Online”or “Offline” using a suitable presence protocol. In both instances, auser may invite any of the “online” or “offline” friends to contributeor collaborate in real time on a project. One invitation may be sent tomultiple users. When a user receives an invitation to collaborate on aproject, the invitation may appear as a pop-up window, or some othervisual that identifies the sender and the nature of the project. Aninvitation may also contain a text body describing the project and whattype of collaboration is desired. A create Mixbook™ button is providedto continue the process of book creation. Invoking button 505 calls up anext screen adapted to enable the creator to select the size (formfactor) of the project.

In one embodiment users may form groups around areas of interest. Inthis embodiment a user that belongs to a group created around a subjectof interest may start a new project such as a photo book relating to thesubject around which the group was founded by clicking on a “start newbook” button on a main page established on the site by the group. Thegroup project initiation automatically included the group members ascollaborators. This represents yet an additional way the communitynetwork is leveraged as way to propagate more project creation.

FIG. 6 is an exemplary screenshot of size-selection page 600 forselecting an initial orientation and size for a project. In thisexample, all of the project options are for multiple page photo books.Other project options may be made available to a user without departingfrom the spirit and scope of the present invention such as a photo card,a poster board, or some other type of publication that utilizes photosand/or and text.

A banner feature advertisement 601 states the intent of page 600 ofhelping the user to select a size for a Mixbook™. The featureadvertisement may include text information (not illustrated) thatinforms the user that the service is free for creating and publishing aproject and that printed materials begin at a specific price range. Auser may also click on “Pricing” at the top of the page to determinewhat the current prices are for printed projects.

Page 600 includes an option 602 for selecting a square photo book thatcan be printed in two available sizes. Although not illustrated, theexact sizes which are exemplary only are twelve inches square and eightand one-half inches square for a square photo book. Other sizes may beoffered for a square book without departing from the spirit and scope ofthe present invention. Option 602 includes information that the book maybe ordered in soft or hard cover. Option 602 includes recommendationsthat are not specified in this example but may include what the systemrecommends selection of a square book is best for such as, best for astorybook (baby), a digital scrapbook, or a photo book that is sharedonline.

Page 600 includes an option 603 for selecting a mini-photo book that isa landscape orientation and that is only available in one printed size.Although not illustrated here the exact size of the mini photo book issix inches by four inches. Option 603 includes information that the bookmay be ordered in soft cover only. Option 603 includes recommendationsthat are not specified in this example but may include what the systemrecommends selection of a mini book is best for such as, family bragbook, portable photo album, or gifts. An additional piece of informationis represented by an information cloud adjacent to the mini-book sample.In one embodiment this additional piece of information may be pricinginformation. The information contained in the information cloud may alsobe some other type of useful information without departing from thespirit and scope of the present invention.

Page 600 includes an option 604 for selecting a landscape photo bookthat can be printed in three available sizes. Although not illustratedhere the exact sizes which are exemplary only are fourteen inches byeleven inches, eleven inches by eight and one-half inches, and eightinches by six inches. Option 604 includes information that the book isavailable in hard cover or in soft cover. Option 604 includesrecommendation by the system that selecting a landscape book is best forwedding photo books, guest photo books, or a photography photo book. Anadditional piece of information is provided in an information cloudassociated with the medium sized landscape sample illustrated in option604. In one embodiment the information in the cloud designates thesample as the most popular size of landscape book. Other types ofinformation may be tagged to the samples using an information cloudwithout departing from the spirit and scope of the present invention.

Page 600 includes an option 605 for selecting a portrait photo book thatcan be printed in one size of eight and one-half inches by eleveninches. Option 605 includes recommendation by the system that selectinga portrait size is best for creating a class yearbook, a team or groupyearbook, or a child's storybook. It should be noted herein that theoffered sizes in the different orientations are not intended to belimiting in any way. A number of other sizes might be envisioned forprint for any orientation without departing from the spirit and scope ofthe invention.

FIG. 7 is an exemplary screen shot of a theme-selection page 700 forselecting an initial theme for a project. Theme-selection page 700includes a scroll menu 701 that contains a number of themes that a usermay select from for use in a project. In typical order, theme selectionmay take place immediately after size and orientation selection whenconfiguring project parameters. Illustrated in the visible portion ofthe window are themes for “Season” including Summer, Autumn, Winter,Spring, Saint Patrick's Day, and Easter. Themes listed under “General”include Classic. Other theme categories might include Wedding, Travel,Baby, Family, Occasion, Holiday, Romance, and Education among many otherpossibilities. In one embodiment a user may create a new theme.

In this example the theme Spring under the category Seasons is selected.Page 700 includes a preview pane 702 adapted to show the theme layoutsuggested for the size and orientation of the project being created. Alayout in the preview pane 702 includes a page 703 on the left and apage 704 on the right. On page 703 there are two photos 707 and one textbox 705. On page 704 there are two photos 708 and a text Box 706. All ofthe photos are those that come with the sample that the user hasselected from menu 701 (sample photos).

A user may select preview theme by invoking option 710 to view a themein the form of the intended product. The user may then select anothertheme from theme pool 701 for preview until the user selects a themethey want. A user may choose a theme by invoking a choose theme buttonon the Website. When the user invokes this option the theme will beincorporated into the project template. A user may invoke option 710,which is an option not to select a theme. A user may also invoke anoption 712, which requests that the system load a project of theselected size orientation and theme with suggested stickers,backgrounds, and photos (if applicable). Invoking the choose themeoption 709 moves the project to the editing stage.

FIG. 8 is an exemplary screen shot of a Mixbook™ Editor (MBE) userinterface (UI) 800 for editing a project in progress according to anembodiment of the present invention. UI 800 includes some of the optionspresented on previous pages including a personalized welcome, an inboxmail indicator, an account option, a help option, a logout option, and alink to a shopping cart. Also present from previous pages are thenavigation links to Home, Create, Pricing, Gallery, and Site Search.

An information bar 802 contains an array of selectable options invitecontributors, change theme, automix, full screen, help, and save. It isnoted herein that the system of the present invention initiates andcompletes auto saves to a user's Mixbook™ project at opportune momentssuch as when the user changes pages within the project when editing. Inthis way the most data a user may lose is data on one project page.

By triggering a save event on specific actions the system mitigates auser's loss of data in the event of software crashes and othercatastrophic events. This system works by persisting the canvas (page)data to the runtime server from the client via a client/servercommunication channel such as an ActionScript Message Format (AMF)remote framework, which transfers AMF data between the client and theserver. This pull system mitigates the amount of work a user couldpotentially lose on a project to just one canvas page worth of data. Italso drastically simplifies the process of creation by removing arecurring step.

In one embodiment, the auto-save feature activates whenever a page isswapped out in the viewing window or “viewport” of the GUI editor.Various other auto-save triggers may be configured and triggeredaccording to established rules. In one example, the auto-save featuretriggers when a specific amount of time elapses while they are workingwithin the editor. In one embodiment the auto-save feature triggers whena pushed event is received by the client that informs the client thatthe site is going down or for any reason. In one embodiment theauto-save feature triggers when a specific copious amount of changes aremade on a single page without save, a threshold established to definethe amount or weight of the changes made.

The automix option in information bar 802 is an automated process drivenby an algorithm. Automix takes the user's photos and dynamically createsnew pages based on the project's theme and the size, orientation, andmetadata of the user's photos. UI 800 has a side bar area 803 thatcontains assets that may be used in creating the project. Sidebar 803 isscrollable and includes at least four interactive presentation controls,each representing a category of assets, which when invoked provideaccess to certain assets under each category represented. The controlfurthest to the left of the FIG. labeled P for photos. Invoking Photosloads all of the user's uploaded photos into the scrollable area for usein the project. The next control to the right labeled L is for layouts.

A layout is a page or spread configuration containing photo slots andtext boxes along with shapes and stickers that go with the theme of theproject. Under layouts, there are sub-categories for project layouts,theme layouts, and recommended layouts. A layout may be a systemprovided layout or one that is created by a user from scratch elements.A next control over to the right is labeled B for backgrounds.

A background is a page background that underlies the added content. Whena background image is stored by the system, the system, in oneembodiment, creates a mirror image of that background by default. Thisunique process is termed background “flipping” by the inventor. Bydefault, all background images are stored as “mirrored images” (originaland flipped version). In this way when a user selects a particularbackground image for two adjacent pages, the flipped image is used asthe second background image and may be auto-inserted onto the page. Forassets that display one or more asymmetric aspects, background flippingprovides a more aesthetic look to a two-page photo book spread. Flippingof assets (storage of mirrored images) may be practiced by default onbackgrounds, layouts, photos, and stickers without departing from thespirit and scope of the present invention.

In one embodiment, an algorithm drives the background flipping processby first identifying a background image that is asymmetrical incharacteristic such as having an asymmetric coloring, texture, pattern,etc. Any images that qualify are flipped meaning that a mirrored-imagecopy of the original is created and stored alongside the original. Inaddition, the system automatically serves a “flipped” image where usefulsuch as when a two-page photo book spread will use the same backgroundimage. The left canvas would contain the original background image whilethe right canvas would contain the mirrored image auto-inserted when thebackground was selected by the user for application to the spread.

The next control furthest to the right is labeled S for stickers. Thesystem maintains clip-art called stickers comprising shapes and otherclipart images used as content to dress a layout on a page. The categoryof stickers includes shapes, project stickers and theme stickers.Project stickers are those that are currently being used in a givenproject and theme stickers are those that are considered relevant to aproject theme. Sidebar area 803 serves as a content interface within UI800 that provides access to all content whether it is system-provided oruser-provided.

In one embodiment the system of the invention includes a recommendationengine driven by several different algorithms that intelligentlyrecommends content according to several different criteria. For example,certain theme-related stickers might be recommended to a user based onthe theme selected for a project. If the project theme is babies, thenbaby related stickers known to the system would be displayed to the userahead of any other sticker types. In another embodiment a hierarchalrecommendation system is provided whereby the system makes suggestionsor recommendations of content based on number of times the content isused by others, commented on by others, ranked by others, and so on.Using a recommendation system helps expedite the project creationprocess by displaying the most relevant or used content assets to theuser first so that the user does not have to search or browse forrelevant content assets to include in a project.

In one embodiment a method for selecting a photo from a plurality ofphotos placed in the project by a project contributor for recommendationincludes (a) accessing the contributor's project photos stored in thedatabase, (b) weighting data associated with each photo used in theproject against one or more business rules, (c) selecting one or morephotos based on the weighting results of step (b); and (d) recommendingone or more photo products containing one or more photos selected atstep (c) to the contributor at an appropriate time.

In a preferred embodiment, stickers such as shapes may be adapted asphoto-viewing windows or “photo-slots” complete with all of the uniqueproperties of a photo-slot such as access to the photo panning tool,photo-edit tool, and other tools for editing. Shapes may also be used astext boxes. There are many possibilities.

UI 800 includes a content storyboard 807 that displays all of the pagesand, or “spreads” 806 of a project in sequence at the current editedstates of those pages or spreads. A spread is simply two or more singlepages connected as a spread of pages. Storyboard 807 is scrollable andmay contain as many pages as are included in any project. A user mayclick on any of the pages or spreads in storyboard 807 to bring it upfor active display and editing. A user may select a viewing mode 808(bottom right) for viewing a single page or for viewing a page spread. Apage tools interface 809 is provided in UI 800 to enable simple pagemanipulations such as adding a blank page, duplicating an existing page,and removing an existing page from the storyboard.

In this example the front cover 801 and the spine 805 of a photo bookare displayed for edit in UI 800. The front and back cover areconsidered pages but are not numbered in this example. Spine 805 is alsoconsidered a page. Interface 800 includes a welcome message 804 that mayappear on display when the user invokes the editing process. The welcomemessage includes a warm welcome to the user followed by some generalinformation about the service and expectations. The pop-up may prompt auser to add photos to the relative project, customize the project, andwhen satisfied with the project, order one or more printed copies of theproject. In one embodiment a themed page already containing stickers andphoto-slots can be added.

In one embodiment a user may remix a project that is in progress orfinished. A page may be remixed by interacting with a remix optionprovided in toll bar 802. Remixing can be undertaken to change thevarious aspects of a page including but not limited to changes in layoutof the page, changes to the background of the page, changes to the fontsof the page, and changes to the styles of items on the page. Remixing apage may also include adding or removing page items like stickers,photos, text boxes, and shapes. Remixing a page may also includechanging effects or the current cropping of items on the page.

In one aspect all of a user's remixing efforts are undoable by clickingon an editing undo button or dialog box. A user may remix any number ofpages any number of times. During the remix operation, a user may beenabled to include photo effects such as B&W, sepia, antique, and so onto all page photos. The user may also be enabled to include meanbrightness and contrast adjustments on all of the photos in the remixand application of font on the page may be included to match as well. Alayout change may source from a pool of layouts stored by the system. Alayout change may also originate from an automated layout algorithm thatautomatically lays out all of the items on a page.

The automated layout algorithm described above may select a speciallayout and then may convert a group of photos into a canvas page. Aspecific layout may be chosen based on the corresponding photo groupdata by the algorithm which further minimizes the amount of zooming orscaling that must be afforded the photos. By minimizing the amount ofscaling required, the resulting canvas page fits the photos into photoslots of similar orientation and subsequently requires very littlezooming of the photos. This lends to higher quality output once a userchooses to purchase a created project in a high-dot per inch (DPI)output.

In one embodiment a flash tool for creating books is embedded in theediting system of the present invention. The flash window expandsautomatically to fill up the user's browser space when the userincreases the size of the window. The flash window contracts to fit aswell if the user decreases the size of the browser space. UsingJavaScript on Resize events, the page creation canvas is adjusted insize to either zoom in to or out of the created book page as the userresizes their browser.

In one embodiment the system may access data quickly for display whileuploading a bandwidth intensive file like a high resolution photo.Bandwidth is a precious resource on the Internet. Photos typically use alot of bandwidth during transfer from one network location to another.While a user uploads photos, the system has the capability of accessingenough of the data using FP 10 to create stand-in thumbnails of theuploading photos so that a user does not have to wait before resumingthe editing process. Once the photos are uploaded, they automaticallyreplace the thumbnails and take on all of the attributes applied to thethumbnails in the absence of the real photos. In this way user will beenabled to work on a group of uploaded photos immediately using thethumbnails as a stand-in.

FIG. 9 is an exemplary screen shot of an add-photo page for acquiringphotos to add to a Mixbook™ project according to an embodiment of thepresent invention. Page 900 appears as a result of the user clicking on“add photos” within the Mixbook™ editor UI. Page 900 functions much likean interface for downloading songs from a music site with the exceptionthat the user is uploading and/or physically referencing photos storedat a remote location accessible to the service provider. An option bar902 is provided within page 900 and includes many separate links topopular network service locations where photos may be acquired for aproject.

At the far left of the options bar, the option upload is invoked. Inthis case the user is uploading photos from a local hard drive or fromsome other digital medium accessible to the operating appliance hostingthe activity. Photos may be uploaded from any internal memory or fromany external memory such as a connected thumb drive, a connected compactdisk, a connected camera, or through some other removable memory thatcan be coupled to the user's computing appliance. In one embodiment thesystem may “capture” photos from media such as a video clip that isplayed back on the user's computing appliance. In one embodiment thesystem may also upload any photos that are scanned into the user'scomputing appliance using any type of image scanning device.

Page 900 includes a status window 901 that is divided into columns androws. From left to right, the columns in status window 901 are labeledFile Name, Status, Progress Bar, and File Size. An unlabeled column atfar right contains either an X or a check mark indicating that theassociated photo is not yet uploaded (X), or that the photo is alreadyuploaded (check mark). There are ten rows illustrated in this exampleindicating that ten photos are to be uploaded. A user may upload onephoto art a time or the user may upload a group of photos by using groupselection techniques such as control click or using a shift and holdoption.

It can be seen in this example that three of the ten photos beinguploaded are completely uploaded into the system as indicated in thestatus column (100%) and by an overall progress bar 903 at the bottom ofthe window. One photo (DSC01273.JPG) is 63% complete while another(DSC01267) is being processed just after upload to determine optimaldisplay size and perhaps other factors for the UI. In one embodimentphoto-add page 900 is a window that appears within the UI of theMixbook™ editor. In this embodiment as the photo status for a particularphoto indicates that upload and processing are complete for the photothen it will display within the side bar area of the UI under thecategory Photos.

Add-photo page 900 includes links for acquiring photos from a pluralityof photo hosting locations on the Internet. The location My Photos isthe collection of all of the photos that the user has already uploadedto the service. The user may click My Photos to view those photos andselect one or more of those photos for transfer to the current project.A user may acquire photos from virtually any Internet-connectedrepository that the user has authorized access to directly or through ahosting service account. Examples of such accounts where photos may belocated include Facebook™, Flickr™, Yahoo™ SmugMug™, Photobucket™, andPicasa™. There may be fewer or more photo-location options (Hosts)referenced within page 900 than are illustrated in this example withoutdeparting from the spirit and scope of the present invention.

If a user has a Facebook™ account that user may acquire any photo thatuser has authorized access to including those photos of family andfriends on Facebook™. The same is true for all of the otherphoto-locations listed in bar 902. A user may, if desired, physicallydownload photos from these third-party-controlled photo-hostingservices. However, the inventors provide a unique data structure forphoto data transfer that permits use of the photo in online and inprinted projects but does not include a requirement for literallypossessing the photo. The data structure represents a photo and enablespartners to transfer photos without performing a physical upload.

The generic data structure includes an array of photo sizes. Each imagesize has at least three fields. These are a value for the photo width inpixels, a value for the photo height in pixels, and the network location(URL/URI) of the image or photo. The address is the address where thephoto is stored. In a preferred embodiment each of the selectable photosizes for a single photo has the same aspect ratio. The term aspectratio refers to the ratio of width to height of a photo. If a partnersite has three sizes for a photo like small, medium, and large, thesesizes must have the same aspect ratio for selectable representation ofthe image without requiring physical upload of the photo.

An algorithm enables the system to determine the best photo-display sizefrom the available partner-configured photo sizes for use in Mixbook™projects. The algorithm loops through the available photo sizes in thegeneric image data structure describing the remote photo. The systemleverages an algorithm for photo display of the remote photo thatautomatically selects the optimal photo size to display out of the arrayof partner-configured sizes provided the photo display on the partnersite. For example, if the service of the invention has three availablephoto sizes (small, medium, and large), and the platform partner, suchas Facebook™ for example, has only two image sizes available (medium andlarge), the algorithm considers the two available sizes in the bestlight. The result may be that the small photo size allowed by theservice would be replaced with the medium photo size allowed for thephoto at the partner site.

The algorithm automatically fits the image into the photo slot. To dothis the algorithm determines the limiting dimension either the width orthe height. To find the limiting dimension, the algorithm takes thefirst element in the size array and creates two ratios. The first ratiocreated is a width ratio which is equal to the width of the firstelement in the size array divided by the width of the display slot. Thesecond ratio created is the height ratio which is equal to the height ofthe first element in the size array divided by the height of the photoslot. If the width ratio is greater than the height ratio, then thelimiting dimension is the height. If the height ratio is greater thanthe width ratio the limiting dimension is the width.

The algorithm orders the sizes by the limiting dimension, which isascending order. Beginning with the first and second sizes, thealgorithm compares the sizes on their limiting dimension with the sizeof the photo slot on its limiting dimension. If the size of the displayslot is bigger than the second size, then the algorithm continues on tothe next size grouping, the second and third sizes. The algorithmcontinues until one of two sizes is greater than the size of the displayslot. At this point, the algorithm will intelligently choose which sizeto display.

The photo is displayed in the same manner as other photos that werephysically uploaded to the service with an exception that the photo isaccessible only when the link from the service to the photo is active. Auser could potentially build and print a photo book that contains onlyremote photos that are never uploaded physically to the service. In oneembodiment option bar 902 is scrollable in the case that more partnersare listed than can be displayed at one time in the window.

The system of the invention includes a partner platform and applicationprogramming interface (API) that enables partners to integrate theirapplications (desktop or web-based applications) with the service of thepresent invention. This platform allows partners to implement codewithin their applications that can transfer data to and from Mixbook™servers including, but not limited to user data, photos, and projectdata (books, calendars, posters, prints, mugs, mouse pads, and otherphoto gift or collage items). In one aspect of the present invention thesystem includes an application program interface API for enablingpartners to access user accounts and make contributions to those useraccounts. Usage of partner aliases helps to distinguish such partnersfrom other users when they are on the site. The system creates an“alias” for each object that a partner application needs to accesswithin the collage editing platform. The created alias will be storedwithin the system to enable quick and easy access to objects stored inthe system. Aliases can be defined by the partner, so that the partnercan use existing data in their own system to reference data in theMixbook™ partner platform. The system will automatically detectattempted duplication of a partner alias. If a partner attempts tocreate a duplicate partner alias, the alias will not be created. Rather,the existing alias may be updated with any change in information passedin the associated action.

In order to create an alias for a given object, a partner must haveaccess through that user by having authenticated access to that useraccount. If a partner does not have an alias connected to a givenobject, there is no way for it to access that object. This makes it easyfor a given partner to create and manage “private” and “secure” datathrough the partner platform without worrying about having othersaccessing that same data.

Partners may create user accounts on Mixbook by passing an alias withwhich they will be able to access the user in the future. If the emailaddress is already registered on Mixbook, the partner must authenticatethe user by sending that user to a Mixbook log-in interface and to theiraccount. Once the user logs in, the alias relationship will be createdwith that user for the requesting partner.

FIG. 10 is an elevation view of a photo spread 1000 of a photo-bookproject exhibiting a dynamically editable text box according to anembodiment of the present invention. FIG. 11 is an elevation view ofphoto spread 1000 of FIG. 10 with the text box moved to a position ofconflict relative to a print parameter applied to a printing process forthe photo-book.

Referring now to FIG. 10, photo spread 1000 is exemplary of any photospread or project page that may be used in a project. In this examplephoto spread 1000 is part of a photo-book project. Photo spread 1000includes two pages whereby the page on the right of the spread includesa text box 1001. Text box 1001 is an asset provided by the system forusers to insert text information relative to project theme and photos.In this example, a user has clicked on the text box 1001 to dynamicallyedit the text within.

A sizing frame 1011 is visible about text box 1001 in this example. Whena user clicks on text box 1001 anywhere on or within the boxed area atext box re-sizing frame 1011 dynamically appears. A text editing toolbar 1003 and a text box rotation tool 1002 also appear insynchronization with the text box re-sizing frame. Frame 1011 enables auser to resize textbox 1001 by grabbing any one of the corner handles orside handles and dragging the handle to change the box footprint. Textbox 1001 can be stretched taller or stretched wider by grabbing anddragging a top/bottom resize handle or a left side/right side resizehandle on frame 1011. Text box 1001 may be made larger or smaller whileretaining the same aspect ratio using any of the corner-located resizehandles on frame 1011. In one embodiment resizing the text box aftertext is inserted into the box results in stretching of the text. Inanother embodiment resizing the text box with inserted text does notchange the font size.

Text box rotation tool 1002 appears dynamically when the user clicks onbox 1001 or anywhere within the box. Rotation tool 1002 may be grabbedand manipulated by the user to rotate text box 1001 about its center tothe left or to the right at any angle desired. In one embodiment anadditional window may be visible that provides the amount of rotation indegrees in real time as the box is rotated by the user. Text editingtoolbar 1003 includes a font style selection menu 1004. Menu 1004contains a field that can be used to type in a known font style or adrop-down feature in the menu may be leveraged to select from a list ofincluded styles available to the editor.

In live editing, an auto-preview feature is provided whereby a user maymouse over a font style exposed from the drop down menu and see the fontchange in text window 1001. When the user slides the mouse pointer offof the selection, the font reverts back to the original font. The usermay type over or type a font within the font window of menu 1004 and seethe font change within text window 1001 once the system recognizes thefont selection. Auto-preview is also provided for other types of editsincluding but not limiting to font size, font color, font presentation(italics, caps, bold, etc), and font justification (left, right,center). A user generally selects or highlights a text item within aproject. This action may cause automatic display of a text editingtoolbar, the toolbar including controls for at least editing font style,font size, font justification, and font color and selecting one of thecontrols of step (b) and mousing over a menu option. For auto-previewinggraphics the user selects or highlights a graphic within the project. Asa result of this action, the system automatically displays a graphicsediting toolbar, the toolbar including a styles interface havingcontrols for applying at least, shadow effect, reflective effect, borderstyle, blur effect, transparency effect, shapes effect, colorsaturation, hue, brightness, and contrast. The user selects one of thepresented controls and mouses over a menu option to preview the option.

Toolbar 1003 includes a font sizing menu 1005 that is adapted to enablea user to efficiently select an optimum size for the font in text window1001. Like the font style menu, the font sizing menu contains a fieldand a drop down list feature. A user may click the downward arrowcausing a font list to be presented to the user. This list contains allof the allowable font sizes. A user may mouse over a font size in thelist and see the size change take place in real time within window 1001(auto-preview). When a user selects the font resizing tool 1005, aseparate slider bar 1006 appears adjacent to the menu.

A user may grab the slider handle of the bar 1006 and move it up or down(vertical slider) or left and right (horizontal slider) to make the fontin window 1001 larger or smaller according to the allowed font sizeincrements that can otherwise be selected or ordered in menu 1005.Auto-preview enables a user operating slider bar 1006 to see the fontchanging in real time within window 1001 as the user moves the sliderhandle. In one embodiment of the present invention, a user may operate aset of directional arrows to change the font size. For example, clickingon the up arrow may increase font size by allowed increments whileclicking on the down arrow may reduce the font size by allowableincrement. Altogether there are four ways of manipulating the size ofthe font in text box 1001. These four methods include moving a slider,clicking sizing arrows, entering a number into a field, and selecting afont size from a drop-down menu. Auto-preview is supported by all of thetext resizing methods and interfaces.

In one embodiment, the system may provide code enabling a three-parttext resizing tool analogous to, from a visual and mechanicalperspective, to font sizing menu 1005 of toolbar 1003 and slider barcontrol 1006. The floating instance of the tool may be created bypulling the menu (1005) off of the toolbar 1003 using a drag and dropoperation. An instance of the tool may also be spawned by linking to theserver-hosted or client hosted .exe file to any area where a user mayinput text such as in a text box, caption box, subject line, dialogballoon, object wrapped text, or any other location in the layout or inan item within the layout that a user may place text.

During an active data session between one or more users involved inworking with the image and/or text-based projects through a graphicsuser interface (GUI), the system aided by software may dynamically orstatically link a three-part text resizing tool to one or moreuser-input actions on one or more text-based assets added to orpre-existing on a page layout. When a user initiates a text input actionin a linked area, the system retrieving and presenting the text-resizingtool in the GUI. In all embodiments, the text sizing slider bar isinvisible to the user unless invoked from the tool bar or when the userattempts to input text in a linked area of the canvas or workspace.

Toolbar 1003 includes a text Bold, Italicize, and Underline tool and aparagraph/justification formatting tool for adjusting the way theparagraph is presented. Anytime the text parameter controls such asBold, Underline, and Italicize are moused over, the user can see theeffect on the text in text box 1001 in real time using the auto-previewfeature. Toolbar 1003 includes a font color management interface 1008that includes three different ways to manage the color of the fontwithin text box 1001.

In one embodiment, individual parts of the multi-part sizing tool areplug-in modules that can be individually linked to and invokedindividually based on link address. In a variation of this embodiment,upon detection of text input into a text box, the slider bar isdisplayed for manual resizing of the input text.

Quick Card Editor

In one embodiment, a scaled down version of the text slider bar 1006 isprovided in a user interface (UI) for editing text in a quick cardeditor. In a quick card editor, which is a lightweight editor, it isdesired that the user may easily change font sizes with a live previewso that they may add more characters to a text box that is limited incapacity (number of characters) at a default or previously ordered fontsize. For example, if a textbox was designed to fit 4 letters into it ata certain font size and then a user wants to add 7 letters into the samebox, they can utilize a simple popup with a horizontal slider to adjustthe font size of the text and receive a live preview of the text as itis shrinking in font size so that they can easily choose the exact sizefor the text to fit the text box.

Interface 1008 includes a rectangular color grid control that calls up athumbnail selection interface containing all of the allowable themecolors in the form of selectable thumbnails when invoked. Interface 1008includes an annular color palette control that calls up a system colorpalette that enables a user to pick a color from a field of blendedcolors or from a color bar of blended colors. Interface 1008 includes acolor picker tool that when selected and or invoked provides a pixelcolor reader to the user's input device in the form of a color dropperthat can be placed over any color manifested on the user display monitorto read a color.

The color of the text in text box 1001 may be changed to any colorvisible in the user interface that is read from a pixel by the colorinput tool. A user may operate the color selection tool to performoperations such as save a color, apply a color to all of a specific typeof item, and so on. Moreover, an extension may be provided to the colorpicking tool to enable a user to blend or mix colors, or to selectmultiple colors in order of priority and save the colors to a personalcolor swatch or palette.

The method can be practiced by selecting or invoking a color readingtool; moving the tool over the display monitor to read and previewcolors using mouse movement; and selecting a color read by the tool. Theselected color may be applied to a single item or to multiple items. Inone embodiment the color picking tool is used to select colors to saveto a color palette or to a color mixing tool. In one embodiment arecommendation system is in use during color picking and can makerecommendations to the user as to the popularity of any color read bythe tool at the time of reading.

Such a category may be segmented according to various criteria such asby collaboration group, by friends of the user, by all site users, andso on. Other interesting statistical data may also be provided to theuser relative to a selected or read color in the form of a pop-up orother visible message. For example the color picker may read a specificcolor and in preview mode a message appears that ranks the color andmentions one or more other colors that go well with the read or selectedcolor. There are many possibilities.

Each of these font color application tools of interface 1008 issupported by auto-preview so that the user may see the text colorchanging before saving the text in a particular color. More detail aboutapplication of colors to font is described further below.

Toolbar 1003 includes a layering tool 1009 containing two controls. Onecontrol brings an item forward to overlay another item and the othercontrol operates to send an item back one layer behind another item inthe window. A unique aspect of interface 1009 is that the system“remembers” what layers items reside on and assumes that all items onthe canvas that do not overlap are on the same layer. The tool will skipintermediate layers if an item is selected to “move forward” to overlapor “move backward” to hide behind another item that is more than onelayer away from the item moved. In this way the user always sees thevisual effect of moving an item forward or sending an item backwardrelative to the other items. The layering system is driven by algorithmand also detects which items are being intersected with and replacesdepth level values attached to those items. In one embodiment, items orassets may be grouped on a page and may be incrementally brought forwardor sent backward as a group of items. Toolbar 1003 includes a deletecontrol 1010 for removing any items on the page or canvas.

Referring now to FIG. 11, text box 1001 has been inadvertently movedtoward the gutter area of the spread by grabbing onto frame 1011 anddragging it toward the center of the spread. The system of the inventionincludes a user alert system for warning a user anytime a text box ismoved to a position where some of the text might be cut off because itintersects or overlaps a print boundary such as print boundary 1100 madevisible to the user as part of the alert process. A visual dialog box1101 appears while the user is in process of moving text box 1001 inreal time. The dialog box contains a warning to the user that themovement of the text box into the print boundary may result in a cut offof text during printing.

The print boundaries are based on the output dimensions for the printedoutput boundaries. The boundaries and text dialog alert box appear onlywhen the user has moved text too close to a boundary edge. A bufferboundary area might be observed in some embodiments to cover outputs inslightly different aspect ratios. That is to say that the warning wouldoccur if a user brought the text box across the buffer boundary eventhough text did not cross the actual print boundary of a particularoutput format. Much work is reduced by providing this early warningsystem way back in the editing process and well before the printingprocess.

FIG. 12 is an elevation view of a photo spread 1200 of a projectillustrating real-time text color application using theme colors. FIG.13 is an elevation view of photo spread 1200 of the same projectillustrating real-time font color change using system colors.

Referring now to FIG. 12, as described further above, font colormanagement may be accomplished using a font color management interface1008. In this example a user has selected the theme colors control ofinterface 1008 as evidenced by the illustrated mouse pointer during aprocess of changing font color of the font in text box 1001. A themecolor grid 1201 immediately appears upon selection of the control. Themecolor grid 1201 contains all of the theme colors 1204 that are genericto the theme the user or the system (in some cases) has selected for theproject. In one embodiment, the theme color grid may include colors thatthe user has previously used in the project or colors that the user hassaved to his or her account on Mixbook. Color pallet 1201 includes acolor swatch containing the current font color of the text in text box1001. An adjacent box 1203 includes the code name for that particularcolor. Auto-preview allows the user to see the font color change as theuser mouses over any of the theme colors before actually committing toany one color.

Referring now to FIG. 13, as described further above, font colormanagement may be accomplished using a font color management interface1008. In this example a user has selected the system colors control ofinterface 1008 as evidenced by the illustrated mouse pointer during aprocess of changing font color of the font in text box 1001. A systemcolor palette 1301 immediately appears upon selection of the control.System color palette 1301 contains all of the system colors 1302 thatare available to the user by moving a pixel reading icon (circle,square, etc.) over the area of the palette to read the colors.

In addition to color palette 1302, a color strip 1303 presents all ofthe same colors from the top to the bottom of the strip. Color strip1303 has a slider bar that a user may grab and drag over the blendedcolor strip to determine a best color for the font. Like theme colorgrid 1201, interface 1301 includes a color swatch 1304 that contains thecurrent color that is being moused over or selected and an informationbox that contains the code name for that color. An interactive option1306 for cancelling a color selection is provided as is an interactiveoption 1307 for accepting the new color for the font in text box 1001.

FIG. 14 is an elevation view of a photo spread 1400 illustrating aphoto-editing frame and intuitive photo panning interface according toan embodiment of the present invention. Photo spread 1400 contains aphoto slot 1407 on the left page of the photo spread. A photo 1408 hasbeen inserted into photo slot 1407 by a user building the project. Photoslot 1407 automatically hides any part of photo 1408 that extends pastits boundaries. The portion of the photo extending past the boundary isinvisible to the user until the user grabs the photo for the purpose ofpanning the photo to improve the position of the photo within the photoslot.

The above process is illustrated by a mouse grab-hand controlled bycomputer input device (mouse). When the user drops photo 1408 onto photoslot 1407 an algorithm automatically sizes the photo so that its top andbottom edges or its left side and right side edges align exactly withthe corresponding boundaries of the photo slot in accordance towhichever dimension of the photo is a closer fit to the correspondingdimension of the photo slot (width or length). In this example the topand bottom edges of photo 1408 align to the top and bottom edges ofphoto slot 1407. A user may then grab photo 1408 in the center using agrab operation and move or “pan” the photo to the left or to the right.The rest of the photo extending past the slot boundaries (left edge andright edge) is visible at some level of transparency as photo boundary1406. When the user lets go of the photo, the portion of the photo 1406is hidden completely providing a convenient and innovative virtual photocropping capability.

In one embodiment the photo slot is an image display window of apredefined geometric configuration disposed on a project page forcontaining and displaying an image that has been placed thereon. In thisexample the user may grab the photo by interacting with an image handleassociated with the image container, the image handle visible onhighlight or selection of the image within the image container ordisplay window. Again if one or more dimensions of the image exceed thedimensions of the image container or photo slot, a user may grab thecontained image by the image handle and may move the image about withinthe container in any direction to present the best view of the image.Parts of the image that are hidden behind the page canvas become visibleto the user when the image is grabbed enabling the user to see whatparts of the image are cropped by the predefined geometry of the imagecontainer as described further above.

A photo sizing frame, similar to the text box sizing frame describedfurther above includes side grab handles and corner grab handles tofacilitate sizing. A photo-slot rotation tool 1409 is provided above thephoto resize frame for rotating the photo clockwise or counterclockwise. Optimal sizing constraints are applied in the algorithm usedto optimally fit photo 1408 to photo slot 1407 such that photo zoomingor scaling requirements are kept to a minimum to increase the efficiencyof the overall process of building a project.

A photo-editing toolbar 1401 is provided to enable a user to make editsto photo 1408. Toolbar 1401 appears along with the sizing frame 1404when the user selects photo 1408 for editing. Toolbar 1401 contains aphoto zoom interface 1402 for enlarging and reducing the photo size.Zoom interface 1402 is a horizontal slider. The photo placed in thephoto slot is optimally sized so that the bar on zoom slider 1402 is allthe way to the left. A user may grab the bar on slider 1402 and drag itto the right to enlarge or “zoom in” the photo. Enlarging a photo causesthe unseen portion of the photo outside the photo slot to expand beyondthe boundary edges of the photo slot. After enlarging a photo, a usermay re-pan the photo to ensure that the best part of the photo isviewable through the photo viewing window or photo slot.

Photo editing tool 1402 includes a rotation tool 1403 that rotates thephoto counter clockwise by an angle of 90 degrees. Photo editing toolbar1405 contains a spread feature for extending a photo over onto the nextpage of the spread. A user may place the photo in the gutter portion ofthe spread and click onto control 1405 to facilitate a view of the photospanning across the two pages of the spread. Another click on thecontrol cuts off the content of the photo that is bleeding over onto thesecond page. In this case the second page is simply the page that thephoto was initially absent from before proceeding to span the photo.

FIG. 15 is an elevation view of photo spread 1400 of FIG. 14 with astyles editing interface 1501 invoked from photo editing toolbar 1401according to an embodiment of the present invention. A user may invoke astyles editing interface 1501 by clicking on a control 1500 asillustrated via a mouse or other input method. Styles interface 1501 isan extension of tool bar 1401. Interface 1501 is also available on textediting toolbar 1003 described further above in the description of FIG.10. Styles editing interface 1501 includes effect application options1502 for enhancing photos.

Options 1502 visited from left to right include a shadow selectioninterface control that when invoked produces shadow levels 1503 from noshadow to heavy shadow. Next to the right is a control option that wheninvoked calls up a reflection interface to apply a level of reflectionto the photo. A border control is provided to the right of thereflection interface and provided a variety of borders to select fromfor bordering photos and text boxes. Next to the border control is ashape control. The shape control provides a menu of shapes (stars,hearts, etc) that can be used as innovative photo crops by dropping theminto a photo-slot. The final styles option is a transparency interfacecontrol that brings up a transparency level interface where a user mayselect from different levels of transparence for a photo.

It is noted herein that all of the same style editing controls arepresent in the styles editing interface for text boxes that can beexecuted from toolbar 1003 described further above. Those controlsproduce the same selections for shadow, reflection border, shapes, andtransparency. Further all of these controls include auto-preview optionsso the user may see the effect in the photo or text before saving theeffects. In addition to the effects listed as options 1502 in styleseditor 1501, other photo effects and editing tools might be included asoptions in the interface without departing from the spirit and scope ofthe present invention such as a brightness control, contrast control,color saturation control, hue control, and other standard photo editingoptions.

Automatically previewing edits to graphics items within the image-basedproject involves selecting or highlighting a graphic within the projectto automatically display photo-editing toolbar 1401 inclusive of stylesediting interface 1501, which may include controls for applying atleast, shadow effect, reflective effect, border style, blur effect,transparency effect, shapes effect, color saturation, hue, brightness,and contrast. A user may select any of the controls and mouse over themenu option to preview an effect.

Before applying any effect to a canvas component like a photo or stickera user may preview how the effect may appear on an object in the styleseditor. The items current effects are considered saved effects. When auser desires to change the effect or style of an item they first selectthe item on the canvas, which automatically opens a toolbar with thestyles and effects editing interface. In one embodiment a display of theselected item is provided in preview mode where the display is of theactual item or a representation of the item.

In one embodiment a user may click on an effect for an item and thenmouse over the item in preview mode to see how the effect will appear onthe item. Mousing off of an item returns the appearance of the item backto its saved state. If the user clicks on one of the preview displays inthe case of multiple displays as in the styles editing interface, thestyles or effects represented by that display will be applied to theselected item and those styles or effects will be saved, such that thesaved state now includes those settings.

Interface 1501 includes a customize option that incorporates a sliderbar 1505 that is shared relative to a plurality of editing optionsincluding Blur; Distance; Angle; and Opacity. A user may select any oneof the just-mentioned options and then use slider bar 1505 to scale thelevel of effect of the selected option. Furthermore, each listed optionhas a text value menu where a user may type in a desired value, orselect a listed value from a dropdown menu. At the very bottom of stylesediting interface 1501, a color editing interface similar or identicalto interface 1008 described previously is provided that includes acontrol for bringing up a grid of theme colors, a control for bringingup a color palette and/or a color strip, and a control for picking acolor using a pixel color reader in the form of a ink dropper thatreplaces the mouse pointer when picking a color. Any pixel that isvisible in the user display may be read and its color may be used torecolor any item of the photo book. At the bottom of interface 1501 aninteractive option is provided enables the user to apply a change to allitems on photo spread 1400.

FIG. 16 is an exemplary screen shot of a contributor invitationinterface 1600 according to an embodiment of the present invention. Thesystem of the present invention enables a user to initiate a project asa creator or project host and then to invite other users to help work onthe project or contribute assets to the project. Interface 1600 appearsas a result of the user clicking on “add contributors” when firststarting a new project as described further above with reference to FIG.5.

Interface 1600 includes a text field 1601 for entering the emailaddresses of other users that the hosting user wishes to invite tocontribute to the project. These other users may already be registeredon Mixbook™ or they may not be community members yet. Users who areinvited to contribute to a project but are nor members of the servicemay be directed to a membership page to register as a user before beingallowed to contribute to a member's project. Interface 1600 includes asecond text field for selecting friends from a list of friends that arealready known to the system. If the friends are already known, the listis presented each time the user clicks on the feature. An option forselecting all of the listed friends is provided at the top right cornerof text field 1004.

Once the invitees are selected, the user may elect to provide a personalmessage to the group of friends invited. A text field 1603 enables auser to type in a text message, which will be sent to the users. Such amessage may explain the project and the reasons for establishing or atleast inviting contributors. For the purpose of term definition as itapplies to this specification, a contributor to a project can be anyperson that is able to contribute creative assets to an existing projecteither in collaboration or singularly. A contributor may be a friend orfamily member already registered to the site. A contributor may also bea family member or friend that is treated as an invited guest that hasnot yet signed up for the service.

Once the contributors are identified either by email address and name orboth, the invitations are automatically sent as email, short messageservice (SMS), Instant Message (IM) or some other invitation message tothose users. Interface 1600 includes a widget 1605 for enabling a userto browse an address book to add users. Once friends are registered withthe site, the user need not enter any email address information toinvite them to be contributors. Contributors may also be collaboratorsthat help the user work on a project in real time. In this case, thecollaborators may already be online as detected by presence reporting ormonitoring. The user simply selects those that are online from the listin text field 1602. An icon may be provided next to the name of a friendto indicate whether that friend is online or offline.

The inventors emphasize that the system of the invention is designed tomake it easy for multiple users to work on and share projects with oneanother in a manner that promotes use of pages from different projectsto contribute to a project in the works. In this way users maycontribute pages from other projects to one they are currentlycollaborating on. An option for making a project open to all communitymembers registered with the site is provided at the start of a project.In some cases projects created by one or a group of community membersmay be copied by another user whereby the new creator edits the originalwork to make it personal to the new user. In this scenario manydifferent projects may contain many of the same photos and othercreative assets that were originally copied.

FIG. 17 is an exemplary screenshot of a proxy-login interface 1700 thatenables a user or contributor to gain access to a personal account tolink to or otherwise acquire access to photos available through thepersonal account. In this example, the login interface is for logginginto a Yahoo™ account where that user may have access to photos storedremotely at the service. Interface 1700 includes the traditional fieldsfor entering a user name or email and password pair. A back button 1702is provided on interface 1700 for navigating back to the previous page.A sign in or login button 1703 is provided for submitting theinformation.

Navigation to the user's personal account takes place through theMixbook™ server by proxy. Once logged in to the account, the systemfinds the photos associated with the account and allows the user toselect and transfer a photo or simply establish an active link to thephoto so that the photo may be used in a project without a physicalupload to the service. This is accomplished by a data structure thatincludes a URL to the photo as described previously. It is noted hereinthat the exact look of the log-in page may vary and may include asign-up option for signing up with the service through Mixbook™

FIG. 18 is an exemplary screen shot of an add friends interface 1800 foradding friends to the service of the present invention. Interface 1800includes a plurality of email services 1801 that a user may haveaccounts with. The user may select any of these services to locatecontacts of the user that the user may wish to invite to become friendson the Mixbook™ site. As previously described the service of theinvention collects a list of invitees and sends emails, IMs or SMSmessages to invite them to the service on behalf of their friend. Oncethe invitees have registered for the service they are included asfriends of the inviting user and members of the Mixbook™ community as awhole.

Once a user has indicated one or more services to access contacts from,the user may select the next button 1802 to proceed to gather the emailcontacts of each of the selected services. In one embodiment thecontacts are in a list resident on the user's computing appliance. Inthe case of Instant Messaging Access Protocol (IMAP) accounts such asGmail for example, the proxy log-in interface of FIG. 17 might be usedto access the contact list for that service.

FIG. 19 is an exemplary screen shot of a preview interface 1900 forpreviewing a finished project according to an embodiment of the presentinvention. Interface 1900 enables a user to preview an online project orother users to view a project they might have access to on the system.In this example the preview interface is in the form of a book that hasmultiple pages with photos and text. In one embodiment a play bookwidget is provided that enables viewing of a photo book in the form ofan interactive slide show or a form correct book preview as isillustrated in the example.

Interface 1900 includes a number of pages in addition to a front cover,back cover and spine (not illustrated). Pages two and three of the photobook are visible and are given the element numbers 1901 (page 2) and1902 (page 3). Referring now to page 1901, a user may configure a slideshow or automated presentation of a Mixbook™ that includes comments suchas comment 1904 made by other users. Comment 1904 includes the user'semail address and time that it was inserted. The system enables users toview their books as an online slideshow that displays all the pages ofthe book along with additional multimedia content that can be added.Users will have the ability to add music or any type of sound track toone or more pages within the book or to the widget to play in thebackground while the book is being viewed as a slide show.

In one embodiment interface 1900 enables automated panning and zoomingof the pages of the previewed book. A small information box 1903 showsthe user that created page 1901 with a very simple statement and emailaddress. If more than one user contributed to a single page then twoinformation balloons might be presented, each citing one of thecreators. It is noted herein that a user may turn off automatic displayof user information related to contributors. Interface 1900 enablesusers who have viewed the project to leave comments embedded on pages ofthe book. In this example a comment is present on page 1901 from a userJoed2 @ aol.com. A profile picture of the commenter is illustrated intext balloon 1904. If many comments are present, they may not alldisplay at once so as not to obstruct viewing.

In one embodiment of the present invention, users may add music orsoundtracks to the system and may associate those to particular projectsor to particular pages of a project. In one embodiment the granularityof adding multi-media content extends to a photo. For example if a photoin a project themed as a barnyard is a rooster, then a multimedia clipof a rooster crowing might be provided and embedded on the pagecontaining the photo of the rooster. When the page showing the user isplayed, the clip of the rooster crowing automatically plays as well. Inanother embodiment the entire slideshow might be set to some backgroundmusic that would play or loop back for the entire time the slideshow isrunning. In one embodiment a static clip associated with a page or aphoto on a page might include a small control panel such as controlpanel 1905. Control panel 1905 enables the user to stop the clip, playthe clip again manually or pause the clip momentarily. In one embodimenta user would be able to interact with items on the page to triggermulti-media “actions” including but not limited to playing a video,playing an animation, moving various items around the page, playing asound or music, or moving to another page in the project.

In one embodiment of the present invention, a preview interface such asinterface 1900 and a slideshow window for viewing an automatedpresentation of the book are two different interfaces. In anotherembodiment both the playback widget and slideshow widget are one in thesame. In this respect the interface “look” may be different depending onwhich process the user is engaging, previewing a book or viewing afinished book in the form of a slide show. There are many possibilities.In one embodiment a photo slot might be associated with more than onephoto such that each photo slot in a slideshow could be a smallslideshow of two or three photos for example.

In an embodiment where a slideshow of a book is to be viewed through asingle presentation window having a footprint smaller than a two-pagebook spread, an algorithm analyzes the content of each page of the bookand dynamically generates a path of traversal through the page contentincluding photos, text, and other media. Page transitions between thelast traversed content on one page and the first traversed content onthe next page may be statically ordered or dynamically selected by thealgorithm from a pool of possibilities.

In one embodiment the algorithm considers for each page of the book, thestate of the previous page and the state of the next page relative tothe location or position of the navigable content and then determinesthe optimal mode of traversing the page using the single presentationwindow used to present the slide show and subsequently determines thebest available transition option to get to the next page. The algorithmcreates a tree of possibilities and selects the best fit path oftraversal for a page and the best transition to the next from the treeof possibilities. The traversal path over a page of a project book maybe defined by the current X and Y coordinates of the window on the pageand the current zoom level of the window. It is noted herein that inslideshow presentation mode, which may be different from book previewmode, the playback widget is able to zoom in and out and pan the contentof the pages during the presentation.

In one embodiment where a slideshow of a project book is viewed, theplayback widget will allow the dynamic display of user comments aboutthe book. While the Play Book Widget is traversing a page, comments madeon that area of the page, or comments that are relevant to a particulararea of a page will be displayed, along with information about the userincluding but not limited to a profile picture, first name, last nameand headline or quote.

In one embodiment of the invention the system incorporates a persistentmessage board for each photo book project termed a book wall or projectwall by the inventor. A book wall allows users to post messages to eachother about an entire project. When a user makes a note on a page of aphoto book the note will be posted to the book wall as well and mayinclude a link to the page. A book wall may be opened from anywhere inthe book. The book wall may be a section that may be navigated to andopened from within a Mixbook™ editor. In one aspect an additionalsection might be created where a user working on a project with multiplecollaborators could propose ideas and suggestions for collaborators.Collaborators may be allowed to comment or edit the creator's comments.

In one embodiment, a project feed such as a Rich Site Summary (RSS) feedmay be provided as a subscriber feed for users to follow certainprojects or collaborators. For example, if a user was working on a pagein a book, they could say something like “I need some help trying tomatch the background with the different fonts”, and then it would appearon the feed that the other contributors are getting as well. Also, wecould display this broadcast of messages to onlookers who are observingthe progress of the book as it gets made.

In one embodiment friends and family members of a user or collaboratorcould get updated project information on their social interaction pagesuch as Facebook™.

Messages could be time stamped indicating the time and date of themessage. Each message could contain information such as the projectidentification and project page the commenter is on. In one embodiment,information relative to the state of the project at the time of acomment is included in forum message, RSS feed or on a project wallLikewise, multi-media screen shots might be embedded in comments fromusers and collaborators. In one embodiment, users might create shortvideo/audio clips that discuss and ask questions from other usersrelative to one or more projects the author is involved with.

In one embodiment, a project wall is accessible from within the project.In another embodiment, the wall is accessible through a separateinterface. Potential collaborators and reviewers could be redirected toYouTube™ to watch videos relative to collaborative comments,instructions, responses, etc. A project wall typically revolves around asingle project. However, a toggle could be provided to switch the feedto include more than one project, or all public projects in process. Afeed could be configured to follow certain individuals or groups as theywork on projects.

In one embodiment the wall is a bi-directional messaging system forselective publishing of text messages with or without graphics. Softwaresupporting the messaging function running from a non-transitory mediumon a server may include at least a function for linking the electronicmessaging system to the image-based project accessible through anelectronic interface, a function for facilitating calling the messagesystem from within the image-based project by executing a link resultingfrom execution of the first function to receive a messaging interface, afunction for detecting and monitoring any online collaborators workingon the image-based project and any online web visitors allowed to viewthe image-based project under construction, a function for assigning foreach message submitted, at least a collaborator identification stamp, adigital time stamp, and a digital location of the collaborator withinthe project at the time the message was submitted, and a function forpublishing the aggregated messages in one or more live broadcast torelevant online collaborators and project visitors.

FIG. 20 is an exploded screenshot of the background selection portion2000 of the user interface of FIG. 8 including an advanced backgroundtouch-up section according to an embodiment of the present invention.The system of the present invention provides a unique color touch-upinterface 2006 for modifying pre-existing or newly created backgroundsthat can underlie page content in a project. Color interface 2006appears when a user clicks on interactive control 2005 labeled more.

An interactive control 2003 for acquiring backgrounds is provided.Control 2003 opens a navigation window that the user may use to browsebackground images stored on the user's system for upload to the serviceof the present invention. The navigation window also provides access toany backgrounds the user has previously saved as personal backgrounds(My Backgrounds) and other backgrounds from other project themes (ThemeBackgrounds).

Many backgrounds might have some textured surfaces like backgroundsdesigned to emulate paneling materials, brick and mortar, wood grain,and the like. Many backgrounds have asymmetric characteristics that canbe flipped so that a pair of background images is created where eachimage is a mirror of the other in the pair.

Color interface 2006 provides advanced background coloring, coloropacity adjustment, and color blending. One unique aspect of colorblending in the system of the present invention is that pre-existingbackground images created by system designers and by some Mixbook™ usersthat exhibit textural characteristics such as wood grains or brickpatterns, for example, can be enhanced with application of pure andblended colors to enhance and personalize them to newer users. Theapplications do not overwrite the original files such as system-providedtheme backgrounds for example.

FIG. 21 is a page-on-canvas view illustrating a relative page coordinatesystem according to an embodiment of the present invention. Interface2100 logically represents a single page 2103 on a canvas 2102. Page 2103is square in this embodiment so it has an aspect ratio of 1:1. Page 2103has a page size that is stored using relative units. For landscape orportrait pages the length of the shortest edge is assumed to be 1 andthe length of the longest edge is equal to the value derived by dividingthe length of the longest edge by the length of the shortest edge. Forexample, if the aspect ratio of a page is 4:3, then the height has ameasurement of 1 unit and the width has a measurement of 1.33333(landscape). The page also uses a relative coordinate system thatoriginates from the absolute center of the page.

The grid pattern of the relative coordinate system may be renderedvisible to a user or may remain invisible to a user. In one embodimentthe grid pattern may be displayed to the user, but at some level oftransparency. The 0 point on the grid is the absolute center point ofthe page.

The coordinate system is used for describing the position, size, andorientation of inserted items like photos, text boxes, stickers, shapes,and so on. Other information about the content like display styles andeffects on those items including but not limited to border, shadow,glow, fill, rounding of corners, reflection, photo cropping, font size,and font style is also stored relative to the position, size andorientation of the items.

Page 2103 has a plurality of elliptical shapes (2104) placed thereongenerally located one per page corner. A star shape 2108 is located leftof center and slightly above the midline of page 2103. All of thedescribed shapes could be photo slots that are used as photoviewing/panning windows. Shapes might be used in other ways on page 2103as well, such as stickers, or text boxes in one embodiment. The pagesize in this example is square so it is described by the relativecoordinate system as −A to +A for width and −B to +B for height.

To establish a relative coordinate system on a page the 0 point orabsolute center is established first. From the absolute center of thepage, the system scales a number of relative units along the widthdirection and height direction from 0 to a negative limit and from 0 toa positive limit for both dimensions wherein a unit is an equal divisionof the total value from the negative limit to the positive limit forwidth and height.

Canvas 2102 includes a sidebar area 2101 containing the options foradding creative assets to the project like photos, layouts, stickers andbackgrounds. Additional shapes 2105 are displayed for selection by theuser. The system is aware at all times of the most current state of eachpage of a project. Logic is provided that enables the system to map thesizes, positions, and orientations of all of the items on a given page.This information is available to the system in real time as a usercontinues to add shapes to page 2103.

A shape 2106 may be a next shape selected for insert onto page 2103 asis illustrated in this example by a broken border and input devicearrow. As the shape is dragged off of the sidebar and dropped onto thecanvas as is illustrated in this example, an algorithm automaticallysizes the shape and places it on the page in an optimum position basedon the real-time data about the current positioning, orientation andsizes of the assets already on the page. The inserted shape is shape2107 having undergone a resizing and reorientation before being placedon page 2103.

In one aspect of the invention a user may auto flow content onto a pageusing a unique grouping algorithm adapted to fit a plurality of photosin the project into logical groupings. The algorithm creates logicalphoto groupings based on timestamp information and other metadatainformation from each photo. Using a wide variety of mathematicaldistributions (i.e. normal, beta), the algorithm determines which photosare related and then logically groups those photos on one page. Themetadata may include date and time that the photos were taken, titles,and other types of data. Additional inputs into the function include,but are not limited to, the average and standard deviation of the numberof photos in a group and priorities and weightings of various metadatainformation for use in grouping.

At various times throughout project creation, display, and outputprocesses, it becomes necessary to convert the relative coordinates inthe page data structure described above into physical measurements. Theconversion process allows the display of the content at any size, eitheron a display such as a computer monitor, mobile device such as a phoneor tablet, or television display, or in a physical printed format suchas a printed photo book or poster. By allowing for the translation ofthe relative coordinates and values into physical measurements, thesystem enables notification to users if any items on a page might notlook acceptable in print. For example, the system can detect if a photowill display at a size in print that is too large based on the size thatwas originally uploaded or added via partner site integrations orotherwise. By storing the data in an abstract data structure withrelative positioning and coordinates, created content can be used acrossa plurality of products and sizes.

FIG. 22 is a block diagram illustrating a grouping 2200 of portrait pagesizes exhibiting similar aspect ratios. An aspect ratio grouping (ARG)is a paradigm or pattern whereby two or more output or print sizes in agiven orientation are grouped together to form a grouping. A grouping ofsimilar aspect ratios for a given orientation includes the availablespecific output sizes that a project may be printed in without requiringany content editing by the creator before print. This innovative aspectof the system enables users to output a project to a print size withinthe grouping that is consistent with the orientation of that grouping.

Grouping 2200 includes three portrait-oriented page sizes in thisexample. These are a page 2201 having a width of L1 and a height of T1;a page 2202 having a width of L2 and the same height of T1; and a page2203 having a width of L3 and a Width of T2. Grouping 2200 comprisespages of portrait orientation. Another grouping is available forlandscape orientation and still another grouping is available for squareorientation. The aspect ratios of each of the pages are very similar. Analgorithm assigns a “core” aspect ratio to a project for display that isformulated as a core aspect ratio value derived from analysis of theaspect ratios of the pages in group 2200. All page content in the sameorientation as an aspect ratio grouping is displayed at the core aspectratio. More particularly, all page content intended to be printed at aspecific page size (say 2201) will be displayed at the core aspect ratioof that specific page size's aspect ratio grouping.

In a preferred embodiment an algorithm is provided for displayingcontent at a core aspect ratio that may be printed to output endproducts like books, posters, calendars, and so on in the sameorientation without any explicit resizing steps or data repositioningsteps required of the user. A canvas described herein as an outputcanvas has a width and a height, bleed areas, and in most aspects, agutter area. The page content is presented online in display duringproject creation using a “core” aspect ratio that is derived fromanalyzing the different aspect ratios in grouping 2200.

The algorithm determines from the canvas measurements what the optimumaspect ratio of the page content will be to ensure a minimum loss ofcontent that would be cut off in the printing process. If the pagecontent aspect ratio is taller than the available output canvas aspectratio, content will be fit to the canvas such that the left and rightedges of the page content fit exactly into the viewable area (left andright edges) minus a gutter allowance. The area on top and bottom of thepage content will be cut off. On the other hand, if the page contentaspect ratio is wider than the available output canvas aspect ratio,content will be fit to the canvas such that the top and bottom edges ofthe content fit exactly into the viewable area. The page content ispositioned on the canvas such that the left or right content edge(depending on which page) is aligned to the gutter edge and content onthe opposite edge which would include the bleed area of the page is cutoff. In this way, page content can be optimally fitted to a variety ofdifferent output formats like two or more different portrait sizes for aportrait aspect ratio or two or more different sizes for landscapeaspect ratio. In general, for an aspect ratio grouping that containssquare-formatted output sizes, the ratio is ˜1. For an aspect ratiogrouping that contains portrait-formatted output sizes, the ratio is <1.For an aspect ratio grouping that contains landscape-formatted outputsizes, the ratio is >1.

The algorithm translates the core aspect ratio of the project to one ofthe selected sizes in a related aspect grouping. For example, a usermight create an eight by six (landscape) pocket photo book using the SWof the present invention. The user may opt to print and purchase a photobook that is a classic landscape size like eleven by eight and one-half.The algorithm places the original page content for each page of the bookonto the output canvas as the correct 11×8.5 classic landscape printsize without requiring the user to edit any of the original content orto manually position the original content on the page. The algorithmperforms the sizing adjustment during the output process in thebackground transparent to the user and no hard changes to stored dataare necessary.

In practice, the system, aided by algorithm, determines for each pagesize in the group, the viewable area of each page, the aspect ratios ofthe viewable areas are recorded. The total amount of error presentamongst the available aspect ratios relative to comparison with a givenaspect ratio may be determined or quantified. Using a mathematicalprogram, process, or automated calculation tool, the effective aspectratio that minimizes the function and hence the total error value isfound. The project page may then be displayed at the effective core oreffective aspect ratio.

In one embodiment, the system determines to “find” an effective aspectratio (EAR) for display during editing. EAR shall be analogous with coreaspect ratio (CAR) in this specification. Both acronyms carry the samemeaning. Given the following three candidate aspect ratios of 1.5, 1.38,and 1.65, the problem is to derive the aspect ratio that would best fiteach of the candidate aspect ratios. In this case R shall equal EAR. Rwill fall somewhere in a range bounded by 1.38 and 1.65. The total errorE can be represented using the following equation

$E = {\sum\limits_{i = 1}^{n}E_{i}}$

where n=the total number of output sizes and where i=1. E_(i) can berepresented a number of different ways. In one example,E_(i)=min(E_(iw),E_(ih)) wherein E_(iw) and E_(ih) represent the errorwhen the effective aspect ratio (EAR) denoted as R in this case, fitsthe height exactly, but spills over in the horizontal direction andwherein the reverse is also true, namely that R fits the width exactlybut spills over in the vertical direction. R will best fit the aspectratio in one of the two ways identified. E_(ih) can be represented by

$E_{ih} = {{\frac{w_{i}}{R} - H_{i}}}$

and E_(iw) can be represented by E_(iw)=/H_(i)R−w_(i)/. Therefore, usingthe original equation

$E = {\sum\limits_{i = 1}^{n}E_{i}}$

we get . . . .

$E = {\sum\limits_{i = 1}^{n}{{\min \left( {{{\frac{w_{i}}{R} - H_{i}}},{{H_{i}^{R} - w_{i}}}} \right)}.}}$

In other embodiments E_(bw) and E_(ih) may be represented in other waysas well. In one embodiment a constant could be added to the derivedequation and the system could multiply through the errors. In this casewe might have an equation like

$E = {\prod\limits_{i}^{n}\left( {{\min \left( {{{H_{i}^{R} - w_{i}}},{{\frac{w_{i}}{R} - H_{i}}}} \right)} + K} \right)}$

where K could be any number>1. In one embodiment the algorithm firesautomatically any time a new product grouping of available output sizesis established. All projects electing a group of output product sizesare displayed at the CAR for that group.

FIG. 23 is a process flow chart 2300 illustrating steps for displayingproject page content at a core aspect ratio and optimizing the contentfor printing at a specified size. As described further above, analgorithm drives the process of optimally sizing and placing contentonto an output canvas of a specific size and orientation for printing.Products that may be purchased and printed are grouped by orientation(landscape, portrait, or square) and by similar aspect ratios thattranslate to specific output print sizes. The content that is displayedonline during editing and content viewing or previewing has a coreaspect ratio derived from the differing aspect ratios of the specificproducts in a grouping.

At step 2301 the aspect ratios of the specific products in an aspectratio grouping (ARG) are analyzed. The core aspect ratio value can bederived by first defining a set list of end product sizes for aspecified aspect ratio grouping. Each end product in the grouping has aviewable page area after allocation is made for the gutter area. Forexample, 0.5 inches may be allocated for the gutter making the viewablewidth area 8 inches for a page width of 8.5 inches. Therefore an 8.5×11inch page has a viewable area of 8×11 inches.

The above definition is made for every product in the same ARG. Once awidth and height dimension of the viewable area for each end product inthe group is established the system can mathematically determine a bestcore aspect ratio for the group. The core aspect ratio is the aspectratio that best fits all of the viewable areas (width and height pairs)in the group. To determine a best fit CAR, in one embodiment, the systemruns sample CAR values against the group values (2301) to quantify meanpercentage overlap errors for the group for each sample value input intothe formula. At step 2302 the algorithm derives a core aspect ratio fromthe analysis of step 2301. The sample CAR exhibiting the smallest meanerror is the best fit CAR for the group. As each end product isconsidered the system determines a percentage of overlap (left/rightand/or top/bottom) for the difference between the sample ratio and theend product ratio. The system squares the resulting error value for eachproduct in the group giving each product an error value and then takesthe mean error value for the group for that sample CAR value. The sampleresulting in the smallest mean error value becomes the accepted CAR forthe group. All content is displayed at this CAR while editing,previewing, or viewing. The data is stored for latter access in step2303.

In an independent thread, a user initiates a project at step 2304(create project) in the same orientation of the ARG of step 2301. Theproduct data in that grouping includes the output format data for all ofthe sizes included in that grouping. An ARG may include two or more thantwo products with similar but differing aspect ratios whereby all ofthose products are of different print sizes. At step 2305 the user MBEobtains the correct core aspect ratio (CAR) from storage for the contentdisplay that was derived at step 2302.

By step 2306 the user has finished the project and places an order forthe project at step 2307. The order will include a specific output sizethat is available in the grouping of products, which are all of the samebasic orientation. At step 2308 the same or a different algorithmperforms a content resize and reposition if necessary for the exactaspect ratio and size of the selected output product. This step istransparent to the creator of the project or to any user ordering theoutput format of the project.

At step 2309 the algorithm loads the page content or “places thecontent” on the output canvas. At step 2010 the project is printed. Inone aspect of the invention printing is carried out by a third-partyservice where the final output is delivered to the printer in a formatthat is print ready so not configuring need be performed at the printer.If no other products are to be printed from the output at step 2010, theprocess may end at step 2012. If the user decides to select a differentproduct to print from the same ARG for the same project, the process mayloop back to step 2307 for placing the new order for the differentproduct included in the ARG product data. Steps 2308, 2309, and 2310 arethen repeated in sequence and the process may end at step 2312 unless adifferent product is then selected from the grouping for print.

FIG. 24 is a process flow chart 2400 illustrating steps for migratingcontent from one aspect ratio grouping to another aspect ratio grouping.The system of the present invention has a mechanism driven by algorithmfor migrating project data from one aspect ratio grouping to another.Project data migration from one aspect ratio grouping to another isperformed in a manner that ensures that no created content is lostbecause of boundary issues in the process.

At step 2401, a user has a finished project created for a specificaspect ratio grouping (ARG) and displayed using a core aspect ratio(CAR). At step 2402 a user decides whether or not the project data willbe migrated to a new project in a different ARG. For example, theoriginal project may be presented as a portrait book and the user wantsto create a landscape book using all of the same information andeffects.

If at step 2402 the user decides not to migrate an existing project, theprocess does not start and simply ends at step 2403 with no actiontaken. If the user determines to create a new project in a different ARGthen the user may select a new ARG at step 2404. As previouslydescribed, an ARG is a grouping of projects of different sizes but thesame page orientation.

At step 2405 the user creates a new project in the new ARG. This isaccomplished working within a project editing interface (Mixbook™Editor). In one aspect an option for migrating project data betweendiffering formats might be made available to users as a static optionaccessible from the Web page or on a toolbar or drop-down menu of theinterface.

At step 2406 the algorithm obtains a new core aspect ratio (CAR) fromthe destination ARG and creates a new project space observing the CAR ofthe ARG. The algorithm then derives at least one multiplication factorin step 2407 for translating the current project data in the originalCAR to the project workspace under the new CAR. The algorithm multipliesthe original project data critical values by the derived multiplicationfactor or factors at step 2408. This may include project data andcontent data including the width and height of the pages, the width andheight of items, the position of items on the pages, and many differentpieces of data for effects on the items on the page.

At step 2409, the content is inserted per results obtained at step 2408according to the new CAR. The result is that all of the original projectdata gets translated to the new format without losing any content due tointersection of content with a print boundary. At step 2410, the systemdetermines if any of the new content has violated any of the printboundaries (print boundary violation (PBV) in the new format. If theprocess runs successfully and no conflicts occur, then the process endsat step 2412. If there is a boundary violation (PBV) detected at step2410, then the process may move to step 2411 whereby the system reportsan error in the translation, which may be conveyed to the user.

In a preferred application if there are any PBVs that occur they aredetected during the translation process and shown to the user in anauto-preview information alert that informs the user of the exact natureof the issue such as what the issue is and what page if not on all pagesthe issue is present in the new format. The algorithm is designed toensure that no violations will occur during the translation of contentto the new format so a determination of a violation at step 2410 wouldbe rare given the controlled environment for translating the data.Auto-preview mode ensures that any error would be detected in real timeduring the translation process and the process would suspend or pausebefore the offending error were realized in the new format giving theuser the best opportunity to resolve the error.

FIG. 25 is a block diagram illustrating system architecture 2500 forinterpolating sample and default data into a project template using aspecial markup language according to an embodiment of the presentinvention. Custom photo-based projects that are theme oriented containtext boxes that are ordered based on a pre-designed project templatesuch as a project template 2501 that is created for a theme by adesigner 2502.

Designer 2502 is connected via network to runtime server (RS) 2509,which may be analogous to AS 109 of FIG. 1 above. Server 2509 has accessvia data network to a data repository 2508 adapted to contain contentsuch as pre-designed project templates, themes, and other componentsthat are integral to the process of the present invention. Any finishedproject such as project 2507 is created from a project template liketemplate 2501. Template 2501 includes a fields list 2511 that identifiesall of the system-provided fields that are part of project template2501. At the onset, all of these fields are blank with no content data,field definition data, position data, location data, orientation data,or size data included. The designer may provide all of the descriptivedata about these fields in template 2501 so that when a project isopened for editing all of the stock fields are located in the properplaces in the project. The field sequence (order of appearance of thefields when navigating the project), and field default or sample text,which are instructions prompting the user to add a specific piece ofinformation to the field when editing a project.

In this example, the designer defines each dedicated field includingwhat type of data should be inserted into each field by an eventual userof the theme-based template. The designer has access to a proprietarymarkup language termed Mixbook™ Markup Language (MBML). Designer 2502may use MBML supported by a MBML (lib) 2503 connected to the designersnetwork to define and dedicate text fields 2511 and to automaticallypopulate end product fields that correspond to these template fieldswith default or sample text. Defining a text field may be accomplishedby tagging the field with MBML. MBML allows designers of templates toinput “default” or “sample” text that will guide the user on what toenter in a given field.

MBML is used in one application by inserting double brackets (“{{”) tostart the MBML, followed by the name of the field being defined.Optionally, the name of the field may be followed by a colon (“:”), withthe “Prompt Text” that will be displayed to the user in a project wizardsuch as project wizard 2504 in order that the user to understand whatshould be entered in each field. The MBML tag may be closed for aspecific data field by double closing brackets (“}}”). It is notedherein that differing syntax may be used to define MBML markup fields orvariables other than double brackets without departing from the spiritand scope of the present invention. For example, in one embodiment theMBML fields could be represented by HTML-type tags rather than doublebrackets. Furthermore, a user-friendly designer interface (notillustrated here) could be provided for convenience as is the case withother markup such as Dreamweaver™ for HTML, for example.

Two or more MBML tags may be associated to one field box withoutdeparting from the spirit and scope of the present invention dependingon what the nature of the definition of that field box is. A date andtime MBML tag pair can be used to mark baby birthdates in a baby photobook. A designer might specify a date and a time using two associatedtags, for example, {{date:Date Baby Was born}} at {{time:Time Baby WasBorn}}. MBML is extensible, such that new field “types” could be definedthat perform other functions, such as inserting a page number, thecurrent date and time, the name of the user, the title of the project,and so on.

In one embodiment, an automatic field population form (AFPF) 2510running on AS 2509 is invoked by default whenever a user invokes aproject template by initiating a specific project. In this embodimentAFPF 2510 takes the appropriate MBML embedded field list 2512 andauto-populates the fields 2513 of project wizard 2504 in display beforea user with the prompt text that tells the user what data should beentered into each field. The user may then enter the appropriate dataspecified by the MBML “prompt” data of each field into project wizardfield boxes 2513, which results in auto-population of the data in thecorresponding project fields 2514 (a-d) of finished project 2507. It isnoted herein that one template field might correlate with two or moreproject fields in a finished project. Repeated fields in a finishedproject that are adapted to contain the same data are represented by asingle field in the project wizard and template.

In this example AFPF 2510 may be generated at the point of the userinvoking project wizard 2504 to enter data into project 2507. AFPF 2510auto populates fields 2513 with the correct prompt text. Another AFPFlike AFPF 2510 may be generated to auto-populate fields 2514 (a-d) infinished project 2507 when the user finishes entering all of the correctdata in fields 2513 in project wizard 2504. In one embodiment anauto-preview feature similar to that described further above is providedto project wizard 2504 that informs user 2506 if any data is enteredincorrectly into any of fields 2513. In another embodiment no projectwizard is necessary and AFPF 2512 auto-populates fields 2514 (a-c) withthe sample or prompt text that informs user 2506 what type of datashould be entered into each field. In all embodiments the entered fielddata are persisted along with the project as part of the project.

It is noted herein that MBML defines variables such as different datastrings that are interpreted as sample or instructional text that letsthe user or the system understand what data is supposed to be insertedin the end project field. A project field shall be defined as anyfeature on a project being edited that may accept text as input anddisplay the text. A project field may include but not be limited to atext box, a title bar, a photo caption, a number field, a date field, anaddress field, a header, and footer. Moreover, such project fields thatmay be automatically populated include those residing in any portion ofany type of project whether the project is in the form of a book, acard, a calendar, a mouse pad, or any other product.

FIG. 26 is a front view of a photo frame 2600 that can be automaticallysized to fit a photo according to an embodiment of the presentinvention. FIG. 27 is a front view of photo frame 2600automatically-sized about a photo. Referring now to FIG. 26, photo frame2600 may be stored using relative size units and may be automaticallysized about photos of differing aspect ratios and sizes. In this viewphoto frame 2600 is not sized for fitting a photo 2601 illustratedherein as a broken rectangle. Typical resizable photo frames have fourstatic components (corners) on the frame that do not change when thephoto is resized. Photo frame 2600 includes four static corners 2604 andfour static ornamental components 2602.

A one image photo frame cannot be expanded outside of the frame'soriginal aspect ratio without stretching and distorting the frame. Frame2600 has expandable regions A and B. Expandable regions A and B areareas on the frame that may be expanded by repeating dimensionally inthe same direction. Added static components 2602 (right and left sides)and 2603 (top and bottom sides) represent ornamental attributes of theframe that would not look right if distorted by stretching or expanding.Therefore, the system determines the current size and aspect ratio ofphoto 2601 and calculates the amount of dimensional expansion requiredin each of the expandable areas of the frame.

Referring now to FIG. 27, photo frame 2600 is automatically fitted aboutphoto 2601 in this example. Static components 2604 (corners), 2602 and2603 are not changed in the auto-fitted frame. Expandable areaspreviously described as areas A and B are illustrated in this exampleafter expansion as areas C and D. In this example frame 2600 has beenautomatically sized to fit photo 2601 with no sizing change required ofthe photo. By adding symmetrical expandable areas on all sides of thephoto frame the system may use more complex ornamental photo frameswithout distorting the ornamental aspects or characteristics of thosephoto frames. The static portions of the frame remain the same in anysize or aspect ratio. Only the expandable areas C and D are changedduring the process. The system detects the photo size and current framesize before the process and then an algorithm determines thecalculations required to produce the correct sized frame for theexisting photo. When auto-flowing or remixing content in a project, theframes accompanying the photos are also automatically incorporated intothe process.

FIG. 28 is an exemplary screenshot of an editing interface 2800displaying a photo-spread including a front cover, back cover, and spineaccording to an embodiment of the present invention. Editing interface2800 is analogous to interface 800 described previously. Interface 2800has a storyboard display 2801 that includes all of the pages of thephoto book being created. The inventors provide a unique paradigm forrelating items across multiple pages including special pages, in thiscase, the front cover 2803, the back cover 2804, and the spine of thebook 2805. When a user edits a spread that contains more than one page,the photos, shapes and stickers can be “overflowed” from one page toother pages within the spread. When the items are overflowing, they arelinked together such that when an item changes on one page, its linkeditem gets updated reflectively on the other page.

In this example the related item is a single landscape photo 2802 thatis spread or wrapped around three special pages namely, front cover2803, back cover 2804, and the spine 2805. In the case of photo 2802, itactively occupies all three designated special pages and editing of thephoto from any of the special pages results in changes to the photoappearing on all three pages. For example, if a user pans photo 2802 theresults of positional change are seen on all of the pages that the photooccupies. If each of the pages has one or more related items that appearon all of the pages for example, and editing performed on one of therelated items may be propagated to all of the related items on otherpages automatically.

Related items may also be moved over to one page completely, so thatoriginal item is not showing whatsoever on the other page. When thisoccurs, the item will be removed from the page it is no longer visibleon. In one embodiment a user may take an item that exists on a lastpage, for example, and may “sweep” the item off of that page toautomatically create a next page with the same layout and theme that ishosting the related item.

In one aspect of the present invention a pages position may be reflectedusing signed integers, which may also be used to model a specialpattern. In this example, signed integers are used to describepositioning of the front cover 2803 (signed integer −1) and back cover2804 (signed integer −2). Other special pages like foldable posterspreads (within a book) and inside cover jackets may also be positionedusing signed integers. There may be a number of different position typesto consider including number (any position value>0), special position,group position, pattern opposition, and function position.

It is duly noted herein that every product available to create at theservice site like books, cards, calendars, and photo books, is acompilation of pages. A page is the fundamental canvas unit observed bythe service although other unit definitions may be observed. A book is acollection of many pages. “Pages” is not just a reference to the insidesof a book, but it can also refer to the cover pieces. The front cover,spine, and back cover are each pages which can be stitched together.

Many times in the process of creating a project, it makes sense toassociate certain pages together. When a user edits a foldable card, forexample, the user may want to be able to move an item from the bottomportion to the top part of the card. From the perspective of the backend data, the foldable card comprises two different canvas units piecedtogether.

FIG. 29 is a block diagram illustrating an asset recommendation systemaccording to an embodiment of the present invention. In one embodimentof the present invention, asset recommendations are made to a user whois editing a current project using the Mixbook™ collage editor. Systemarchitecture 2900 includes a user 2901 that is currently online withruntime server 2904 running an asset recommendation engine (ARE) 2905.The system of the invention has access to all of the assets that havebeen designed by system designers, users whom may be registered as assetcontributors, and any other user contributions to the system. Creativeassets 2903 include all of the stickers, shapes, backgrounds, layouts,photos, and any other accessible imagery.

When a user edits a project, and selects a theme, theme-based assetsbecome accessible to that user's Mixbook™ editor as illustratedpreviously in FIG. 8. In the canvas-editing interface (MBE), one cancreate a hierarchical recommendation system for presenting users withcontent to use in their current and future projects. By using themes asthe building blocks for this hierarchal asset recommendation system, wecan then link a theme to a project and utilize the creative content ofthe project as the basis for the theme.

In one embodiment, a project can be used by the system as a basis fortheme creation in order to facilitate creation of more projects based onthe theme. This theme creation process can be leveraged by bundling allforms of creative assets from a user-created project into a theme. Ifthe creator of the project so chooses, all of the assets in the projectcan serve as the basis for a theme, which then serves as the creativebasis for creating future projects. For example, if a user creates a“Mother's Day” project in the form of a photo book, they can use thisproject as the basis for a theme called “Mother's Day Theme.” This“Mother's Day Theme” can then serve as the basis for users across thesite to create their own “Mother's Day” projects. They will be able toutilize the canvas pages and all other creative assets that were addedto the original “Mother's Day” project that was used to create thetheme.

In one embodiment the assets are related to a specific theme a user hasselected for a project. Creative assets like backgrounds stickers,layouts, and so on appear in the interface as thumbnails along with theuser's photos, etc. In one embodiment asset recommendation engine 2905is adapted to list recommendations by relativity. For example, if a userlike user 2901 is working a project that is baby themed, then thestickers relating to that theme would appear in the user's editinginterface. For example, hierarchical recommendation may take the form ofsticker categories 2906. Categories 2906 include “My Stickers” as thetop level of stickers (the first to appear in the interface). “Mystickers” would include the stickers added directly to the project bycontributors to the project. “Theme stickers” would next appear andwould include the system-provided theme stickers that go with thatparticular theme. “Recommended Stickers” may include other themestickers used in the projects from the themes that this project's themeshares a theme category with (Baby Themes).

Creative assets may be recommended on a contextual basis as well. Forexample, if a user is editing a specific page that has baby stickers onit and clicks the sticker tab other baby stickers will be shown to theuser, including the ones used on that page. Other recommendation schemasmay be practiced in accordance with the present invention withoutdeparting from the spirit and scope of the present invention.

A cloud labeled All Users 2902 may represent all of the users currentlyediting projects, or all of the users registered with the system. Assetsmay be recommended to user 2901 based on what all of the current usersor based on what all of the users have determined to be the best or mostused or most popular assets like theme-based stickers. In creativeassets 2903 clouds 2709 represent possible recommendation genres. Forexample, if the asset is stickers, a recommendation might be the mostpopular stickers based on use count among all users. Another categorymight be the most popular stickers based on use count of only thefriends of the user receiving the recommendation. Another category mightbe recommendation of the stickers that might best fit a specific project(contextual). Yet another category might be a recommendation of creativeassets based on bandwidth considerations. There are many variedpossibilities. Any type of creative asset might be considered forrecommendation using contextual or hierarchical recommendation using ARE2905. Recommendations may be made to users according to analyzed usagecounts, friend relationships among the content creators, and contentcharacteristics like width and heights or content meta-data such asphoto data.

In one embodiment a single asset recommendation system analogous to ARE2905 is provided for recommending creative assets stored by the systemto users editing a project or a project page. Assets may include but arenot limited to the categories of stickers, shapes, backgrounds, layouts,and frames. In a preferred embodiment the asset recommendation system isimplemented in software and integrated with the GUI for recommendingcreative assets. In one embodiment the asset recommendation engine is aback-end data process that works in the background to rank creativeassets and serve relevant assets according to ranking order. In thiscase each creative asset considered has a score card that may reflect afinal score or value based on one or more conditions or “weightings”.

The system is adapted to receive server calls relative to user GUIactivity and to parse information about creative assets stored for use.A ranking function is provided that ranks creative assets based onfiltered and/or or unfiltered statistical data known about usage andcharacteristics of the asset input against one or more pre-set rules forranking the appeal of that asset. The system includes a server functionfor serving the recommended assets into the GUI. In this way the mostrelevant assets may appear in the interface closer to the top followedby the lower ranked assets and so on per asset category. The process isdriven by algorithm and may fire each time the user clicks on an assetcategory to browse available creative assets stored by the system of theinvention.

Both hierarchical and contextual recommendations can be made in concertto users operating the GUI. When a user clicks on an existing project toresume editing, the project's creative assets are loaded into theproject pages as described further above. These creative assets are alsolisted in hierarchical order in the asset selection interface, typicallythe sidebar area of the editing GUI. In one aspect the hierarchy is MyStickers, followed by Theme Stickers, followed by Recommended Stickers.The ranking portion of the engine ranks My Stickers or “ProjectStickers” as being most relevant to the project because they are alreadyestablished in the project by the project contributors. Therefore, thesealready used stickers will be stickers that have the greatestprobability of being re-used in the project.

The ranking engine ranks theme stickers as the next level of importancebecause these stickers are designed to be used in the theme that theuser's project falls under. Now, the ranking engine looks at stickersthat are stored that have been used in other user-created projectssharing the same theme or belonging to the same theme category but thathave not yet been used in the instant project. The system ranks thesestickers according to a weighting algorithm that calculates the appealof each qualifying sticker according to one or a combination ofparameter constraints.

In one example, the recommended stickers sent to a user for display inthe recommended stickers' window might be the highest ranking stickersbased on a use statistic filtered by user type. For example, this user'sfriends prefer the following recommended stickers based on the number oftimes they have been used in projects sharing the same theme or even thesame theme category. On the other hand, the system may rank the stickersbased on some other preset condition that is relevant to the situationat hand or may rank stickers based on likeness to stickers alreadyselected for use in a project or on a project page.

In one embodiment the asset recommendation requirements are a receivingfunction for receiving server calls relative to user GUI activity beingmonitored, a weighting function for ranking creative assets based oninput against one or more rules for ranking asset appeal, and a serverfunction for serving the recommended assets into a GUI. In simplepractice of the invention includes (a) receiving an indication of a userselection of a project or project page for edit at an assetrecommendation engine; (b) calculating or updating current scores forthe assets that qualify for service based on project, project page, andtheme-based characteristics; and (c) serving the assets into the GUIasset browsing windows with the most relevant assets based on analyzedscore served first followed by lesser relevant assets.

In one embodiment, the system of the present invention can select one ormore photos from a user's project for recommendation in a new product.The system stores the in-depth page positioning information for all ofthe items placed in a user's project. The metadata about each of theuser's photos, for example, is persisted along with the in-depthpositioning information with each item stored for use in the project.

The system may therefore make intelligent recommendations regarding theimportance and relevance of photos and other assets to a user. Throughthis analysis, the system can make automated suggestions of single ormultiple photo products that the user may have an interest inpurchasing.

By analyzing asset usage information and other data the system canaccess the items stored for a specific project that may contain, forexample, hundreds of photos and select one photo or a few photos thatwould qualify for application to single or multi-photo products likecanvas prints, coffee mugs or mouse pads and so on.

Referring now back to FIG. 29, the system may use ARE 2905 to make anassessment of which image or photo assets in a product are goodcandidates for isolation for a single or multi-photo product. The termmulti-photo refers to a product with two to several photos, for example,where the user had many photos in the project pool. A coffee mug mayinclude a photo wrapped around the mug or several photos positionedabout the mug. There are many different possibilities.

The ARE may first access the creator's project photos stored in thedatabase. Then the system might weigh data associated with each photoused in the project against one or more business rules. After weighingthe results, ARE 2905 may select one or more than one photo based onweighted score. In one aspect each item has a score card. The system mayfinally suggest or recommend one or more than one photo product as an upsell at such time the user wishes to make a purchase. A unique aspect ofthe recommendation is that the system may present one or more views ofthe suggested product already loaded with the creator's photos selectedby the system.

ARE 2905 analyzes any tags attributed to photos and the descriptions ofand captions associated with those photos. ARE 2905 might consider theactual size of images for certain product types like canvas prints, forexample. If a photo is a background that spans two pages in a photobook, it may be a good candidate for a canvas print. ARE 2905 canconsider the number of times a photo is used in the same project. Therecommendation may be updated as a user edits a project such that when auser is ready to purchase the project in print form the system mightalso recommend that the user purchase another product that can bedisplayed to the user through the GUI with the recommended photos inplace on the product. The system might offer an up sell option of one ormore than one product exhibiting the selected photos.

FIG. 30 is an architectural diagram illustrating a printing network 3000according to an embodiment of the present invention. Printing network3000 includes a user 3001 connected online to a runtime server 3002.Runtime server 3002 is analogous to runtime servers previously describedin this specification. Runtime server 3002 includes a digital mediumcoupled thereto and adapted to contain all of the SW and data requiredto enable server function.

When user 3001 is editing a project using a Mixbook™ editor (MBE) suchas MBE 3006, for example, the user is connected in session with server3002. As such, the user may decide to print a finished project such as aphoto book by invoking a printing option such as print project option3008 illustrated in exploded view. Such an action invokes a printingengine 3007 that is responsible for rendering the user's project in aready state for printing according to a final end product selected bythe user.

The printing engine sends output files for printing to a third-partyprinting service 3003. Printing service 3003 represents any onlineprinting service. Third-party printer 3003 includes a printing server3004 and a connected printer 3005 representing printing capability.Printing engine 3007 is a flash based that is enhanced to produce highresolution output files to print. Print engine 3007 is a single printengine that is involved from the point of creating a project to thepoint of printing out the project.

A flash-based print engine has some drawbacks that need to be overcomein order to print larger high resolution files. Outputting large formatprints for printing from a flash based editor is very difficult to do.The inventors provide enhancements that enable the system to take flashcontent and produce a near pixel-perfect reproduction of a user's workin the high resolution formats that professional printers normallyrequire. Traditionally print systems take the back end data and thenrecreate the content for printing through a second engine such as aJava-based print engine or a C-application. Print engine 3007 is a pointfor point engine and can re-create content at any dots per inch (DPI)required by the printer.

In one embodiment gutter areas of printed products are optimized forview after print. In most situations a traditional printed book having agutter area will have content sucked into the gutter area where it ishard to view, or there will be a color strip applied to the gutter areawhereby the color strip appears somewhat unprofessional. Print engine3007 functions to solve the above problems by optimizing the size of thegutter area and content displayed on a page before printing. Inparticular, the system of the invention implements a unique method ofgutter size calculation and content display logic for printableproducts.

Based on other content optimization algorithmic results, a guttercalculation algorithm calculates the best size for the gutter based inpart on the aspect ratio grouping. The gutter area is intelligentlysized in order to minimize the amount of content that is not viewable onprinted book products due to the glue that lies in the gutter betweentwo pages on a given spread.

The gutter sizing algorithm works in conjunction with a content sizingalgorithm to balance the two extremes, one being no gutter allowance,and the other being too much gutter allowance. No gutter allowanceresults in content hidden by the glued area in between pages. Too muchallowance causes the area defined as gutter to be displayed visibly tothe user.

FIG. 31 is a process flow chart illustrating steps for printing largeoutput images according to an embodiment of the present invention. Atstep 3101, the system pulls a project for print from a print queue.Users that order printed products are taken through an ordering processafter which their print jobs are queued for servicing. A project forprint may be a calendar, a wedding book, a baby book, or any photo-basedproduct with one or more pages. At step 3102, the system invokes aprinting engine analogous to Flash print engine 3007 describedpreviously. At step 3103 the printing engine accesses or otherwiseacquires the project for print. Pages of the project are the canvasesthat contain the photos, stickers, shapes, and text. A book spine isalso considered a special page as described earlier in thisspecification.

At step 3104 the print engine processes the project pages and encodesthe pages for printing. In this step the print engine reads the page andattempts to render the page as a snapshot encoded in some image formatlike a bitmap image, PNG image (Microsoft), or a jpeg image format.During processing of any of the project pages the system determines atstep 3105 if there are any problems with the process so far like anylimit errors regarding overall page size or weight. This step isrequired because there are specific limitations using a Flex/Flash-basedprinter that prevents snapshot taking of images at higher resolutions orif the snapshot is allowed it results in a huge drain on computingresources (low memory error). In most instances any content sizing orresolution issues with the individual photos in the project pages areresolved through a print warning system described further above that ispart of the MBE. Therefore, any issues with any of the atomic elementswithin the project like content and print boundary conflicts should havebeen resolved before pre-processing for print begins.

Typically, in a generic process an output image would be renderednormally and then a snapshot of the image would be taken for print.However, quality would downgrade with larger images printed using anon-optimized printer and printing process. Likewise, scaling down theimage or image content on a page would reduce print quality. Forexample, a 12×12 image for print at 300 DPI (desired resolution forquality in print) would require roughly 3600×3600 pixels. If at step3105 the system determines that there are no errors the system mayoptionally embed the page images into a PDF document for printing fromthe PDF format at step 3109. The output image could be embedded intosome other application like MS Word without departing from the spiritand scope of the present invention.

It is noted herein that the print engine prepares all of the pages inbatch before releasing the materials for printing. The system thendetermines at step 3110 if the pre-printing process has been completedfor that project. If the system determines that the page pre-processingis not complete for all of the pages of a project, then the processfalls back to step 3104 until all of the pages are pre-processedincluding special pages like the front cover, back cover, spine, insidejackets, and any other ‘special’ pages within the project. When thesystem determines that all of the processing is complete for all of theproject pages, then the print engine releases the project as a print jobto be sent to the end printing system at step 3111.

If at step 3105 the system determines that there are one or more limiterrors preventing the print engine from taking a single snapshot of theentire page, the offending page may be tiled into two or more imagetiles of smaller size at step 3106. The print engine may be adapted toperform this task. In one embodiment this task may be passed on toanother application for tiling and stitching photos. The smaller tiles(snapshots) created in step 3106 may be saved to memory by the printengine as uncompressed raw byte array (RBA) files for each page in agiven project that was required to be split at step 3105. In oneembodiment an image tiling module is provided and integrated with theprint engine. The print engine may automatically sense if the pixel sizeof an image is greater than a pre-defined image size threshold for highresolution printing and if so tiles the image and saves the individualimage tiles to memory for later reconstitution.

In a preferred embodiment, the print engine is adapted to detect imagesthat require tiling by algorithm. In one embodiment, the algorithmcalculates the threshold for any image submitted for printing byleveraging a mathematical formula that includes the size (width andheight) of the print area of the image and the limit value for the printengine to take a single snapshot. In a preferred embodiment the printengine is a Flash print engine. In one embodiment the threshold isexpressed in dots per inch (DPI). Using the algorithm, a 12×12 imagewould have a DPI threshold of about 340 DPI over which tiling would berequired to maintain a good quality print. For a 14×11 inch page, theDPI threshold would be about 330 DPI. It will be understood by one withskill in the art of mathematics that there may be more than onemathematical approach that could be implemented to derive the thresholdlimit value for determining if an image for print needs to be tiledwithout departing from the spirit and scope of the present invention.

The smaller tiles created at step 3107 are reconstituted back into asingle page image at step 3108 using an outside image stitching tool.Optionally, the process resolves back to step 3109 where the page imageis embedded into a PDF file for print. In one embodiment, PDF files aregenerated by the print engine with layers and vectors. The print engineis able to rasterize certain portions of a PDF file while generatingother portions of the file in vector to retain the best print qualityfor page content. The process requires embedding of fonts.

At step 3110 the system determines if processing is finished for all ofthe pages in the project. If the system determines that processing isnot complete at step 3110, then the process jumps back to step 3104. Theprocess typically runs for a batch if there is more than one page imagein a project.

With a normal effects filter like a shadow effect filter, flash willprevent the effect from appearing about the page image or other contenton the page image like photos and shapes at higher resolution requiredfor quality printing. For example, if a user has a large full-page photowith a drop shadow effect around it the effect would not be visible atall at higher resolutions required for printing. Other effects aretreated similarly such as glow, reflection, and so on. Filter effectswill not be visible if the filter area is greater than about 15,000,000pixels. The system enabled by the print engine provides a smallerversion of each ordered filter and renders those into bitmap images.This bitmap image (one per filter) is then scaled up to the filtersperceived size.

In one embodiment the printing system may tile the background of a pageimage by mirroring it on all edges to create a buffer zone around atrimable area of a printed project. If book trimming ever invades thisarea the mirrored effect is considered more aesthetic than viewing ablank white strip around the edges. In one embodiment of the presentinvention users may configure a project to print using one or moredimensional ink printing techniques on certain content contained withintheir projects or on whole page images. In one embodiment certaindimensional ink printing effects may be preordered by the system bydefault.

In one embodiment the print configuration engine is implemented insoftware and is integrated with the Mixbook™ editor (GUI) and includes acommunications link to a compatible print server and printer connectedto the data network. In one embodiment tasks are deposited in a printqueue. An effect filter size reduction module is provided for reducingthe filter size. In this case, the print engine automatically senses viathe effects filter size-reduction module if the area of a filter isgreater than a pre-defined threshold and if so produces a smaller filterembedded into a bitmap image after which the bitmap image is scaled backup to the filters original size.

FIG. 32 is an elevation view of a photo book project 3200 undergoingediting for dimensional printing according to an embodiment of thepresent invention. Dimensional ink printing is a relatively newtechnology that enables three-dimensional effects to be produced inprint relative to photos, shapes, stickers, borders, and text contentwithin a project. More particularly, dimensional printing allows variousareas on the printed canvas to have a “raised” or “3D” look to them.Dimensional printing today is generally accomplished by using a KodakNexpress™ printer, and providing a secondary map to the original imagewhich instructs the printer on which areas to “rise” and by how much.

While the end printer is enabled for three dimensional printing, theprint engine in conjunction with the Mixbook Editor™ creates the threedimensional print instructions or “maps”, which are layered over theassociated content to instruct the end printer what areas should beraised and by what depth to print the effects. In a preferredapplication all products are configured for print and then sent to athird-party printing service to be printed out.

Referring now to FIG. 32, project 3200 is assumed to be presented withinthe Mixbook Editor™ (MBE) of the present invention. Project 3200 is aphoto book in this example, but could be any other type of project suchas calendar or poster without departing from the spirit and scope of thepresent invention. Photo book 3200 includes a page containing one largephoto 3201 placed in a photo slot as evidenced by the panning controlcircle in the center of the photo. As described further above, when auser clicks on photo 3201 editing toolbar 3203 immediately appears toallow a variety of edit options including calling up an additionalstyles interface 3202. Styles interface 3202 includes an option forpre-configuring photo 3201 to be printed using dimensional ink toproduce a three-dimensional (3D) effect on the printed version.

In this example, styles editing interface 3202 includes various editingoptions including an option labeled 3D for dimensional printing. A usermay invoke interface 3202 by clicking on an icon 3203 in the automaticediting toolbar 3202. In this example, a user clicks on 3D (DIMPrinting) to invoke an algorithm that “reads” the photo color histogramand/or perhaps other photo data and creates several thumbnail images ofphoto 3201 for preview whereby each thumbnail exhibits a different stockdimensional print effect as it would look on photo 3201. In thisembodiment the default raised levels may be previewed in styles editor3202 visually in the preview panel. The user may select or mouse overany thumbnail in the auto-preview panel to see the effect on photo 3201.

It may be difficult to “see” the raised effect just by looking at athumbnail. Therefore, in one embodiment color that is indicative ofareas that will be raised by default in each preview thumbnail isapplied to each of the thumbnails to point out in each case where theraised areas are. The intensity of such a color may indicate degree ordepth of the 3D effect applied.

In one embodiment the algorithm changes every pixel in the “raised” areaof photo 3201 to produce a shadow effect. In this case areas that aremore raised will appear lighter in color and the areas that are lessraised or not raised will appear darker. Areas with more height willcast shadows across the areas with less height using the shadow effect.Other methods to denote to a user that specific areas of an item havebeen raised are also possible.

A project may be configured simply by displaying in the editor, a pagecanvas for editing; selecting or highlighting an image or item on thepage canvas, invoking a dimensional print effect configuration interfaceby selecting an interactive button on a toolbar displayed automatically,or by implementing keyboard shortcut to execute the interface, andselecting an option for applying one or more three dimensional effectsfrom the interface. In one embodiment dimensional print settings includeautomatic generation of a three dimensional print effect whereby theautomatic process is responsible for analyzing color data of the image,consulting one or more rules relative to the analysis of color data, anddetermining from the consultation which areas of the image should beraised and to what depth. A depth map is generated for the image, thedepth map instructing a printer which portions of the image to raise andby what additional depth to achieve the three dimensional print effect.

In one embodiment, a method for configuring a project for threedimensional printing includes displaying in the editor, a page canvasfor editing, selecting or highlighting an image or item on the pagecanvas, invoking a dimensional print effect configuration interface byselecting an interactive button on a toolbar displayed automatically asa result of the previous item selection or by implementing keyboardshortcut to execute the interface, and finally selecting an option forapplying one or more three dimensional effects from the interface to theitem. The effects may be automatically previewed in the editor.

In one embodiment a user may further click on customize to open adrop-down section that may contain a slider and one preview pane. Insuch a case, a user may be enabled to select pre-defined or mappedportions of a photo by mouse input and then operate the slider tointensify or detensify the effect (raise or lower) the depth level ofdimensional printing associated with the specific area of the photo. Thealgorithms used to adjust content for dimensional printing rely on aseries of pre-defined rules that govern what items may be dimensionallyprinted and how items are analyzed for “best fit” dimensional printing.In one embodiment, the subsystem parses a selected item and thensuggests or recommends a style and depth of dimensional printing for anitem like a photo for example.

When a user accepts a dimensional printing option for an item thealgorithm creates a depth map for the image page or photo that isapplied as a layer over the photo image. Depth map options might includehaving a higher raise at the edges, having a higher raise on one edgevs. another edge, raising certain areas of the image, etc. In oneembodiment the system denotes to the user which areas of an item likephoto 3201 will be raised by application of one of several possibleinterfacing methods like applying 3D shading to the item, a glowingeffect, an icon next to the item, a highlight, outlines, and many moreoptions. Project templates that include stock items like photos shapes,stickers, and text boxes could be created by designers to includedefault settings for dimensional printing applied to individual ones orall of the items available with the template.

In another embodiment an algorithm intelligently selects which areas ofa page or item should be raised in print to produce an optimum 3D effectbased on one or more pre-defined rules. When a user selects auto-raisefor example, the algorithm may analyze all of the items on the pageincluding shapes, stickers, photos, text frames, borders, and so on.Aspects subject to analysis include, but are not limited to colors used,photos used, text entered, numbers of items on a page, colordistribution patterns in the product layout, color distribution patternsin inserted photos and so on. Analysis may also consider size or aspectratio of an item; the depth of the item on its canvas in relation toother items; size or aspect ratio of surrounding items; and any effectscurrently applied to an item or surrounding items such as drop shadowand its associated properties; or otherwise comparing the item withsurrounding items.

After a page is analyzed, potentially considering some or all of theanalyzing points mentioned above, the algorithm chooses the best areasto raise and the best depths to raise the pixels in raised areas. Thealgorithm may then indicate to the user as described further above whichareas will be raised in preview. The user may ultimately have the optionof accepting the “system recommendation” or not. Various methods couldbe utilized to determine areas that should be raised over other areas ofa photo, item, or page. The system might determine the most common coloron a page and may raise all of the pixels exhibiting that color to aspecified depth. The system stores the “height” of various items on aper-pixel basis. A dimensional attribute is added to the normal pixeldata so instead of red, green, blue, and alpha (opacity/transparencyratio) (RGBA) the pixel is defined as RGBA and pixel depth (D). Everyitem having raised pixels on a page will have its own depth map thatwill document the data so that it may be transmitted to an end printeroperated by a print partner for end fulfillment. The depth D may bedifferent for pixels in separate raised items and for pixels within asame item.

Rules for auto-raising elements of a project might include raisinglighter areas of a texture over darker areas. All text of a specificfont type might be raised. Rules may be pre-configured for each type ofitem, for example sticker rules might be different than shape rules.Areas within stickers might be raised based on color distribution withinthe sticker. Frames and borders may be raised by default. Theauto-dimensional print configuration algorithm will rely on the rules togenerate depth maps for any items containing raised pixels. In oneembodiment user experience may be fed back into the system to helpdesigners create new default applications for dimensional printconfigurations.

In addition to enabling efficient utilization of assets available withinthe system, designers pre-build projects from starting projects. When auser starts a project, the system saves the starting version of the bookto the database. Many copies of the project are then prebuilt for users.When a user begins a project, we start them off with a book that looksexactly like the project that the designers created (minus the photos).The resulting project is prebuilt so all the system has to do is updatethe foreign keys to reference the projects new owner, making the processvery efficient to the end user. Another unique optimization isencouragement of copying projects.

The system allows users to create a copy of a project and allow the“copier” to modify all the contents of that project and (optionally) useall the photos in the new project. Remixing a project means to take acurrent project and create another digital copy of it on the system. Inthis way new users can quickly and efficiently make an existing projecttheir own. The publishing interface portion of the Mixbook Editor™ hascontrols that allow users to gauge the level of collaboration, viewingaccess, and “copy” rights of their projects. The system enables “open”projects where any other members can view contribute and copy.

Response to Monitored Pre-Transactional Behaviors

In one embodiment of the present invention, the system is enabled bysoftware to track and monitor client navigation history, including realtime navigation of the service online. Among other uses for the datagleaned through pre-transaction behavioral analysis is to determine whenit is appropriate to incentivize clients to make purchases in a mannerthat is good for business and the client. Many current systems haveincentive programs for clients, but these are generic in nature and comein the form of coupons or standing discount offers communicated to allclients as opposed to direct targeting clients that are statisticallymost likely to engage in a transaction based on the summation of theirbehaviors. In one embodiment one or more purchasing triggers areidentified and defined as rules. Such rules may be stored for service ina data repository associated with one or more servers of the system.

FIG. 33 is a block diagram depicting an incentive determination anddelivery network 3300 according to an embodiment of the presentinvention. Network 3300 includes the Internet network, denoted herein bynetwork backbone 3306 and any connected sub-networks.

The system with the aid of the software 3307 may track and reporttraditional aspects of an intelligence model for a client such asgender, age, profession, etc. More importantly, when the clientoperating a computing appliance, such as appliance 3311 is online andengaged in interaction with the service, the system may track and reportbehavioral patterns or tendencies, and navigation patterns andtendencies each time the client accesses the service. In a preferredembodiment, the data collected is maintained in a client repository 3304or section of memory in a database. The system may choose to run ananalysis on a client, most preferably when the client is online, usingthe service, and can logically engage in a transaction. Data previouslytracked and recorded and stored in repository 3304 may be pulled fromstorage along with fresh data taken while the user is currently engagedwith the service.

In one example, a client may be working on an image-based projectconsisting of wedding invitation to the client's wedding. Along withother data, the system may note and store the wedding date. When theclient returns to the site after the wedding date, the system mayperform an analysis of the client data on record to determine if it isappropriate under the current business model to incentivize the clientto purchase a photo wedding book. Such a wedding book might be currentlyin process or might not yet have been initiated by the client. Otherdata such as the client's transaction history might also be tapped tohelp determine an incentive amount to offer the client for agreeing totransact with the system for the wedding book. If the client hascontributed steady revenue to the site over time, the system may usethat data to help determine a monetary amount to offer the client.

In practice, once an active session is underway with the client, amonitoring and recording function captures the working behaviorsincluding content contributions of all or individual ones of the usersoperating through the at least one GUI. Working behaviors may includebut are not limited to site navigation, click pattern, socialinteraction, content contribution, and transacting. The data may beanalyzed against a set of incentive rules stored in a repository 3305.Incentive rules may be based on an incentive model for clients. A server3301 is provided and connected to backbone 3306. Server 3301 hosts SW3307 adapted to monitor and track client behavior (navigation, purchasehistory, actions, etc.) of individual ones of clients or of clientgroups collaborating on a project. SW 3307 tracks and monitors userbehavior while in the domain of the system, such as while creating andediting a project or visiting other projects, etc. SW 3307 alsodetermines for each client if there should be an incentive-based offer,such as offer 3312 sent to and displayed on computing appliance 3311relative to a network interface (NI), which could be any connectednetwork application including an active user interface 3310 used tocreate, edit, and order projects like a wedding book, for example.

In determining whether or not to initiate an offer, SW 3307 relies onclient information held in repository 3304 for input along with anyfresh data collected but not yet stored. An algorithm runs the inputvalues for a particular client against incentive rules 3305. Theanalysis may lead to generation of a score or an active threshold value.The threshold value or score resulting from analysis may help determinea yes or no answer relative to offering the client some incentive towarda purchase based on the collected and analyzed data. In one embodiment,the software determines, after analysis, a specific time window duringone or more subsequent and active data sessions (client connected toservice), that the client should be incentivized to purchase aparticular product based in part on analysis of all or a portion of datarecorded. SW 3307 dynamically determines a monetary amount for theincentive based at least in part on the collected pre-transactionaldata. This determination could be made based in part on the parametersof the particular product identified for most likely or possiblepurchase and in part on the results from analysis of the data recordedbefore and during the current session.

Once it has been determined through analysis that the client shouldreceive an offer such as offer 3312 tied to a purchase and what valuethe offer should be worth to the client, the system providesnotification (3312), at a dynamically determined point in time duringthe instant active session, or at some point after the session. Offer3312 may be a cash incentive such as five dollars toward the purchase oftwo wedding photo books, for example. The client may receive such anoffer via an instant message (IM), a short message service (SMS)message, an email, a multimedia message (MM), a computer screen pop-upmessage, or a verbal cue or prompt through the network interface (NI) ifso enabled. The offer could be an added credit, a discount, or someother offer of value to individual ones or to the group of users of theavailability of and the amount of the cash incentive.

In one embodiment, pre-transactional data 3304 may be pulled andanalyzed by SW 3307 against rules 3305 at such time when the client isnot online. It is not required that the client be logged into theservice to receive an offer like offer 3312. A messaging server 3302 isillustrated in this example and may be an email server or a serveradapted to handle other types of text messaging such as short messageservice (SMS) or instant message (IM), for example. Messaging SW 3308 isdepicted as running on server 3308. In this embodiment, SW 3307, withthe aid of server 3301, may contact and utilize messaging SW 3308 todeliver a pre-prepared message containing an offer for the clientrelative to one or more projects. A runtime server 3303 is illustratedin this example and represents any server or group of serversresponsible for hosting UI 3310 enabling clients to access theirprojects and perform other activities relevant to the overallexperience. Server 3303 is connected to a graphics repository 3309 thatcontains all of the project related data including graphics, assetcontributions, and data. It is noted herein that assets may be stored inother network locations as long as they may be accessed by the system onbehalf of the client.

It is noted herein that pre-transactional data collected about a clientmay include historical behavior, transaction history, and demographicinformation. Client behavior may include navigation paths, clickpatterns, published comments made by the client, the types of projectscreated by the client, and other tasks that can be tracked monitored andattributed to a client while the client is interacting with the site. Inone embodiment, SW 3307 may be enabled to map or compare a client'sbehavioral patterns to patterns determined for other clients whom haveused the service in the past. Value determination may also be tied tosuch mappings or comparisons. For example, if X number of clients allhave a similar pattern of sending out wedding invitations, initiating awedding photo book, but have not sought to purchase the books, a highercash incentive might be offered to all of those clients than what mightbe offered to just one or two clients. Notifications (offers) may bedelivered to clients while they are interacting with the service orafter the fact when they are offline or otherwise not engaged with theserver.

Creation and Management of Fractal Images

Referring now back to FIG. 21, shapes 2104 may be photo slots that areadapted to contain images or photos. The terms image and photo shall bedeemed synonymous in this specification. In one embodiment the systemwith the aid of an algorithm may produce a fractal image effect from asingle photo placed into any one of photo slots 2104. In one embodiment,this may be accomplished through a right click action on any photodropped into any photo slot 2104. In another embodiment, a pull downmenu command or a keyboard command may accomplish the same function. Inthis case, a fractal image is shared by photo slots 2104 with each slotdisplaying its own portion of the entire image.

The algorithm runs automatically for any photo that a user decides touse in a fractal image display using two or more photo slots. In oneembodiment, the system determines how many photo slots of what shape andsize to create on a canvas to accommodate the fractal parts of the photoselected by the user for the fractal image effect. In anotherembodiment, the client or user determines the shape, size, and locationof the photo slots on the canvas used to display the fractal imageeffect.

For panning, when you move a photo in one of slots 2104, thecorresponding positions of the photos in the remaining photo slots alsomove in the same direction along the X, Y coordinates. This gives theeffect that there is only one continuous photo being displayed. In thealgorithm, code binds the fractal items (separate portions of the photo)together based on X, Y coordinates. When a user moves one photo in oneslot 2104, translation instructions are extended to the other photoslots to reproduce the same movement with respect to their fractal viewsof the photo. It is noted herein that this effect may be performed for asingle photo (different photo portions displayed in each slot) or forseparate photos (different photo for each slot). For example, if a photoin one photo slot is moved by an X, Y position, then the SW translatesthe same movement to the other items in the fractal group. In oneembodiment, other actions besides panning can be translated to all ofphoto slots such as zooming in, zooming out, stretching, resizing,rotating, and application of special effects.

In one embodiment, animation may be applied to a movement so that themovement is repetitive and automated when the photo canvas is beingdisplayed. In the event of printing, the final print version capturesthe last displayed positions before print of the photo items in theanimated embodiment. The algorithm may display parts of an image in allof the slots in a fashion as to capture the entire image using all ofthe slots. In this case, no part of the original image displayed in oneslot is reproduced in any of the remaining slots. Boundary limits mayalso be imposed by the algorithm to ensure that movement in one slotdoes not result in another slot displaying, for example, the edge orborder of an image. Other operations are possible such as zooming by anamount in one slot resulting in the same zoom amount to be translated tothe items in the other slots. Zooming in by an amount in one slot mayalso translate to the reverse effect (zooming out) on the other items.In still another embodiment photo slots may be assigned functionsrelative to displaying their fractal parts of an image or of a group ofimages.

Simulated Force Dynamics Between Digital Assets

Referring now to FIG. 29, runtime server 2904 includes a physicssimulation engine (PSE) 2907. PSE 2907 may be any physics simulationengine adapted to deliver motion dynamics resulting from detection of atrigger relative to one or more objects or items that have beenconfigured for interaction such as having interactive boundaries thatattract or repel objects with polar or similar boundaries. There aremany such simulations engines available for rendering motion dynamicsfor digital objects that are assigned certain properties and includetrigger criteria such as having interactive boundaries. In oneembodiment, items include any digital graphics that may be added to aproject canvas such as stickers for example.

FIG. 34 is a screen shot 3400 of a photo book project that is open anddisplaying two pages, generally referred to herein as canvas. Canvas mayinclude any workspace for rendering graphics. In this example, there isa first canvas page 3401 and a second canvas page 3402. Each pagecontains at least two photo slots for containing digital photos. Theseare photo slots 3403 and 3404 on page 3401 and photo slots 3405 and 3406on page 3402.

In this embodiment, a client has arranged digital items 3407 (horse) and3408 (horse) in opposing positions on top of a bed of grass 3413. Horse3407 and horse 3408 are facing each other. Each horse includes aninteraction boundary, boundary 3409 for horse 3408, and boundary 3410for horse 3407. In this example, a soccer ball 3412 is provided with aninteractive boundary 3411. In this case, boundary 3411 triggers motiondynamics for soccer ball 3412 when it comes into contact with anotherboundary it recognizes like boundaries 3407 and 3408. Grass 3413 is notinteractive and has no interactive boundaries other than providing astable floor for the interaction between the soccer ball and the horses.However in another embodiment, grass 3413 may have an interactiveboundary.

In this embodiment, a user clicks on horse 3408 and moves or drags thehorse along the grass until it interacts with soccer ball 3412(boundaries touch). Thus, motion dynamics for soccer ball 3412 aretriggered, namely that the ball rolls or bounces away from horse 3408.The motion dynamics may be calibrated according to the nature of thecontact, for example a soft touch or a harder touch may affect the speedand distance that the ball moves away from horse 3408. Horse 3407 ispositioned in the path of soccer ball 3412 facing horse 3408. Whensoccer ball 3412 collides with horse 3407, it is repelled back in thedirection from whence it came according to the force of the contact(boundaries 3410, 3411). The motion dynamics are such that the ball iskicked back and forth by the horses on top of the grass.

The motion is initiated in this case by a client selecting one of thehorses and making moving it towards the ball until the boundariesintersect. In this case the boundaries are repellant boundaries. Thelighter weight of the soccer ball (physics attribute) causes the ball tomove and not the horses resulting in the desired effect. The motiondynamics are simulated by a physics engine as previously describedabove. The items (horses and ball) are linked together algorithmicallywith respect to the boundaries and desired physics effects. In thisparticular example, the items and interaction between them span bothpages of the editable canvas 3400. However, items and interactionbetween then may be confined to a single page. Moreover, boundaries mayrepel or attract relative to motion, or may trigger some otherpreprogrammed physics when making contact with another recognizedboundary. An item may have a boundary that does not interact with aboundary of another item.

FIG. 35 is a process flow chart 3500 depicting steps for configuringitems for interactivity according to an embodiment of the presentinvention. The process begins on the system side with step 3501 whereinan administrator selects digital items for configuration. Theadministrator assigns interactive boundaries and their properties to theitems at step 3502. At step 3503, the administrator defines the triggermechanism, in this case, intersecting boundaries, and the resultingmotion dynamics that are desired. Once triggered, an algorithm maydetermine the motion dynamics and in more complex interactions, aphysics engine may be employed. The process may be aided by or may beperformed by software (SW) in a semi-automated or wholly automatedfashion. At step 3504, the administrator or SW stores the items forclient access when online and working on a project.

In the case of SW, the SW includes a function for establishinginteractive boundaries between items in a collection of items placed onan editable canvas, a function for detecting, upon movement of at leastone of the items, the intersection of one or more of the interactiveboundaries established, and a function for dynamically implementingphysics assigned to the items upon confirmation of intersection of theinteractive boundaries. The latter function may rely on a physics enginesuch as one typically used in video gaming for rendering physics.

At step 3505 a client assumed to be online and working on or editing aproject accesses items for building onto the project. In someembodiment, interactive items are already part of a template and may bekept, removed and replaced with different items. For each item accessed,the client may determine by observation, or may receive visualindication which items selected are interactive and which other itemscertain items may interact with. The client may therefore decide at step3506 if items are interactive or not. Typically, step 3506 occurs beforeitems are added to the canvas at step 3506, if the client determinesthat the item or items are not interactive, the client arranges theitems on the canvas without regards to animation at step 3507. Theprocess then skips to end at step 3510 for those items. In oneembodiment, items may be grouped according to interaction and may bepresented to users in such groups or item sets.

If the client determines that the items selected are interactive witheach other or other items at step 3506, then the client may investigatethe exact nature of the interactivity including review of anysub-options for editing the motion dynamics if allowed by the system atstep 3508. The exact nature of the interactivity of an item might begarnered by right clicking on the item and selecting interactionsummary. In one embodiment, a client may further expand the summary toproduce more granular details and options if they exist for a particularitem.

At step 3509, the client may arrange the items on the canvas accordingto the interactivity data to ensure that the motion dynamics workcorrectly on the canvas. The process may then end for those items atstep 3510. In one embodiment, users may be empowered to access a buildkit to create items with motion dynamics and interactive boundaries orpoints. It is noted herein that motion dynamics plays out only while theproject is being edited or reviewed online connected to the runtimeserver. The online version of the project may contain many animationsand motion dynamics between objects or items on each page.

Managing Print Order Distribution Among Multiple Printing ServiceVendors

FIG. 36 is an architectural view of a service network 3600 supportingmanagement of print order distribution according to an embodiment of thepresent invention. Network 3600 is, in one embodiment, an Internetnetwork. Network 3600 includes a runtime server (RS) 3607 for hosting anetwork interface for users to use in creating image and or text-basedprojects like a photo book, for example. Server 3607 includes at leastone processor and a data repository, the server including anon-transitory physical medium.

Network 3600 includes connected users, illustrated herein logically asuser or client network 3602, labeled All Users. Network 3602 includesany communications network available for connecting computing appliancesto the service. A computing appliance 3604, a desktop computer in thisexample, represents a user finishing a project 3605 while connected toruntime server 3607. Project 3605 represents any printable project thatmight be created by a client using the service. One objective of theservice is to take print orders from users for printing and delivery(typically by shipping) to those users that ordered and paid forproducts.

RS 3607 has connection to a data repository 3608 adapted to contain userproject data presented in a finished project for printing. It is notedherein that some project data may be sourced or stored elsewhere on theInternet including user-registered sites such as social interactionsites and photo bucket-type sites not affiliated directly with theservice. RS 3607 also has connection to a data repository 3609 adaptedto contain active print orders submitted to the service for printingsuch as print order 3610. All print orders are prepared for printingusing a print engine 3611. Print engine 3611 formats and otherwiseprepares a project for submission to one of multiple available printvendors operating throughout the network. Print order 3610 is receivedby RS 3607 and prepared for printing by print engine 3611 before it isstored or queued in repository 3609. In one embodiment an activefirst-in-first-out queue system is provided for queuing print orders fordistribution. In this example, such queuing is represented by repository3609.

Network 3600 includes a routing server (RTS) 3613. Server 3613 includesat least one processor and a data repository, the server including anon-transitory physical medium. Routing server 3613 has connection torepository 3609 containing print orders for processing. Print orderscontained in repository 3609 have already been prepared for printing byprint engine 3611. Routing server 3613 includes software (SW) 3614running from the non-transitory physical medium. SW 3614 includes afirst function for acquiring current job capacity capabilities, deliverytime estimates, historical production times, and current cost estimatesfor printing and shipping relative to one or more products resultingfrom completion of the one or more electronic print orders. This firstfunction may be performed as an ongoing or periodic task. Routing server3613 has connection to a data repository 3603. Repository 3603 containsprint vendor data such as current job capacity, delivery time estimates,historical production times, and current cost estimates for every printvendor available to the service.

Network 3600 has access to an online print vendor network 3601 in thisembodiment. Vendor network 3601 is a logical network representing thenetwork paths and network access to all print vendors available to theservice. In this example, print vendors 3606 (1-n) are represented bynetwork cloud icons. Each print vendor 3606 (1-n) has an applicationprogram interface (API), the collection represented as APIs 3615 (1-n).Print vendor APIs may be shipping and production status APIs adapted tointegrate with SW 3614 to provide the necessary information for theservice to determine which print vendor is the best one to handle anyparticular print order or batch of print orders according to economicsand logistics that may be favorable to the service. The determination ismade algorithmically using one or more than one algorithm. Therefore,print vendor data 3603 is constantly being updated with older obsoletedata being continually dumped or discarded with the exception ofhistorical data. SW 3614 includes data routing software for dynamicallyrouting one or more of the electronic print orders according to theresults of scoring of the print vendors available to the service.

Intelligent Capacity Planning

SW 3614 includes one or more algorithms providing a function applyingscores associated with the combined values for each of the printvendors, and then comparing those scores among the available printvendors in determining which vendor is currently the best vendor to usefor an order or batch of orders. In a use case example, assume that eachavailable print vendor 3606 (1-n) guarantees a particular productioncapacity that the service can utilize on any given day of orderfulfillment. Looking at cost factors, printer 3606 (1) guaranteesprinting a particular product for ten dollars whereas print vendor 3606(3) guarantees printing the same product for only five dollars. If costwere the only criterion, routing server 3613 would route print order3612 directly to print vendor 3606 (3). Conversely, if more data isevaluated and compared between the vendors such as current productioncapacity, and consideration of a second product (print order) ready fordistribution, the determination may be different.

Print vendor 3606 (1) may guarantee printing of a second product typerepresented by an additional print order at four dollars compared to tendollars for the second product if printed by vendor 3606 (3). Capacitydata may indicate that vendor 3606 (3) is only one print order away fromreaching it's maximum printing capacity for the day, the service mightselect print vendor 3606 (1) to print the first and second products,keeping print vendor 3606 (3) open if another print order is predictedto be ready that day that may be routed to that vendor. The servicesaves one dollar and reserves print vendor 3606 (3) for the nextpredicted order. In one embodiment, print vendor data repository 3603includes pre-negotiated rate tables submitted by print vendors in theauspices of competition.

In one embodiment, SW 3614 may also access customer history data forcustomers ordering printing services for their created projects. In suchan embodiment, customer loyalty and historical activity may garner somepriority in selection of an available print vendor for printing an orderfor that particular customer and shipping the finished product to thatcustomer. The concept here is that if a customer can receive a printedproduct sooner than later, the customer may experience a higher level ofcustomer satisfaction that may lead to further patronage.

In one embodiment wherein SW 3614 is integrated with the print vendorAPIs (3615 (1-n), or where pre-negotiated cost tables are available, theservice has intimate knowledge of the up-to-date costs to ship productsto the customers from any given print vendor. However, greater customersatisfaction that might result from more expedient shipping resulting inmore expensive shipping costs could yield a margin back to the serviceover a reasonable time period that is greater than the originalinvestment in the more expedient shipping of product. In this waypredictive analysis may play a role in calculating which print vendorsto use for particular customer orders. In this embodiment, routingserver 3613 and SW 3614 has access to historical customer records suchas purchase history and activity history at the service site. A customermay be identified as one who would likely create and purchase moreproducts in a reasonable time because of enhanced satisfaction resultingfrom more expedient shipping of a last order. A threshold score could beestablished and customers exhibiting a level of activity that surpassesthis threshold value could be candidates for enhanced shipping whereinadditional revenue generated over future orders more than makes up forthe increase in shipping costs laid out by the service.

Real-Time Bidding by Print Vendors

In one embodiment, print vendors 3606 (1-n) may compete with one anotherfor business by adjusting their prices according to an auction modelgoverning printing costs. In this embodiment, a portal might be providedfor the print vendors to log into to adjust their printing costs in abid to pick up more immediate business from the service. The servicecould institute maximum constraints based on the pre-negotiated pricesfrom each print vendor. For example, if print vendor 3606 (2) has someextra capacity because of a slow period or day, that print vendor couldlog into the print vendor portal supplied by the service and manipulatea slider bar that may produce different stock keeping units (SKUs)describing product types and costs. One component of integration may bea preview of added business that could result from a modest change inpricing. Therefore, as a print vendor operates the slider bar, thecorresponding increase in volume of print orders they might receive atthose adjusted prices would also appear in the interface.

FIG. 37 is a process flow chart 3700 depicting steps for routing a printorder based on analysis of data. At step 3701, the system receives aprint order over a network, the print order submitted by a customerthrough a graphics user interface (GUI). At step 3702, the system withthe aid of a printing engine prepares the project subject to the orderfor printing by one of multiple available print vendors accessiblethrough the network. Once the print order is prepared and optimized forprinting, the print order is queued for routing of distribution to oneof the available print vendors at step 3703.

At step 3704, the system gathers current print vendor data such ascurrent job capacity capabilities, delivery time estimates, historicalproduction times, and current cost estimates. The system then analyzesthe data for each of the available print vendors and compares the scoresto determine the best print vendor to handle the print order. At step3706, the system assigns the best print vendor as the recipient of theprint order. At step 3707, the system routes the print order to thedesignated print vendor. The process then resolves back to step 3701 fortreating the next print order. It is noted herein that a best printvendor may be determined for a ready print order or for a batch of readyprint orders without departing from the spirit and scope of the presentinvention. A batch of print orders may contain projects that arestructurally and graphically similar, or projects that are differentfrom one another without departing from the present invention. Likewise,a batch or ready print orders may be split and distributed to more thanone print vendor as determined by data analysis and scoring withoutdeparting from the spirit and scope of the present invention.

Intelligent Reverse Invoicing

In one embodiment, chart 3700 includes a step after step 3707 orgenerating and sending a reverse invoice to the print vendor assigned tothe print order at step 3706. For example, if the print vendor confirmeda price for a print order at 9 am, but changed the price for a similarprint order received at 11 am, it would be possible for the vendor tomistakenly charge the 9 am price for the later order. By invoicing theprint vendor at assignment of the vendor for an order where the vendorhas confirmed the price at the time of the order, the system ensuresthat the correct invoicing prices are adhered to. In this way thesystem, with the aid of software running from a non-transitory medium onthe server may secure a current price from dynamically changing pricingfor an electronic print order among a grouping of two or more printvendors operating over the network.

In one embodiment, the current pricing associated with the print orderrouted to the assigned vendor is recorded for subsequent data populationon a generated digital invoice covering the print order includingshipping, etc. The invoice is sent electronically over the network alongwith the print order to the vendor. In one embodiment, time stamps ortime tags may be used as a mechanism to confirm the current pricing atthe time of routing of an electronic print order to a selected printvendor. The time stamp or tag can be used at the vendors end to doubleconfirm the appropriate pricing that was in effect at the time the orderarrived in the vendor system.

Prioritizing Graphical Digital Assets

With reference to FIG. 29, ARE 2905 may leverage a number of statisticalgenres to help prioritize creative assets that are presented tocustomers for possible inclusion into their projects. Network 2900supports a statistics server 2908. Statistics server 2908 includes atleast one processor, data repository, and a transitory medium. Server2908 hosts SW 2909, labeled Activity Monitor. Activity monitor 2909 maybe hosted on runtime server 2904 and may be integrated with ARE 2905without departing from the spirit and scope of the present invention.

ARE 2905 may also communicate with a separate activity monitor 2909 asshown in this example distributed to server 2908 to better aiddiscussion of separate functions. Activity monitor 2909 is adapted tomonitor activities of customers and non-registered site visitors for thepurpose of gathering data to create useable statistics across multiplecategories relative to the activities of customers and, in someembodiments, relative to the activities of non-service-registeredfriends and family members of those customers who may visit the site tolook at projects or otherwise browse published projects created by thosecustomers. Activity monitor 2909 may monitor all of the activityoccurring within the service domain, such as at the service Website (notillustrated), and during project creation, editing, and transactionprocesses through a graphics user interface (GUI).

Server 2908 has connection to a data repository 2910 adapted to contain,at least, client navigation and usage statistics. In one embodimentnavigation and usage statistics may be gathered from non-registeredusers such as friends and family of registered users who visit theservice domain such as a Website, portal, social interaction page,project publication page, and so on. Activity monitor 2909, in thisembodiment, works in the background independently from ARE 2905 tomonitor all users and to collect data about those users and activitiesthey engage in at the service site domain. Activity monitor 2909includes a processing component for creating statistics formulated underseveral categories. One such category is usage of assets such asrepetitively used content contributions, themes, stickers, frames,styles, etc. Another possible category of data relates to a useractively liking or disliking an asset or contribution. Site navigationpatterns of users through the service domain represent another possiblecategory for gathering data and forming statistics. Still another datacategory is sales data relative to projects created at the service, forexample, the current best-selling themes.

Activity monitor 2909 creates statistical information about data it hascollected and store the information in data repository 2910. As activitymonitor 2909 creates new statistics, the data in repository 2910 isupdated and older data that is obsolete under any category may bepurged. A time window for keeping data may be imposed as some categoriesmay fluctuate widely in the amount of and relevancy of statisticsdepending on season, trend factors, or other like factors of change. Forexample, the theme “Mothers Day” and related creative assets and usagestatistics would not be relevant during the time just before “FathersDay”, but become relevant again around the next “Mother's Day” season.

In prioritizing creative assets, the system, which may leverage ARE2905, attempts to present to the customer the best-fit assets the systemcan offer for what the customer's goals are in the way of projectcreation. ARE 2905 may run for each user who is actively creating and orediting a project. In one embodiment, ARE 2905 may communicate with acustomer, making pre-project recommendations before the customerinitiates a project if there is some indication of the intent of thecustomer. When ARE 2905 is called to recommend assets to a user, it mayaccess data repository 2910 to obtain statistics relative to a theme,theme category, layout, and associated individual assets like stickers,photos, etc. that may be presented in prioritized fashion for that user.If the best, most likely accepted items are presented to the user first,less work will be required of the user to create a project.

The system may determine, for example, which existing template would bebest for the user by looking at statistics formed from sales data andnumerical analysis. In one example, assume that there are two photosarranged on a user's page for remixing. A category for data gatheringcould be data about history of all purchased books having layouts with 2landscape photos arranged on them. Candidate layouts then may beprioritized further to narrow the field down to a few layouts that wouldbest serve the customer. Template or layout categories may be filteredfurther by narrowing the selection criteria based on related themes tothe current project being worked on.

Prioritized assets may be presented to a user in a sidebar of thegraphics user interface. Activity monitor 2909 includes at least afunction for assessing and recording the usage statistics of digitalassets used by the overall client base. Activity monitor 2909 alsoincludes a function for monitoring and recording the navigationbehaviors, usage patterns, and content contributions of all orindividual ones of users operating through the at least one GUI orvisiting the service domain. ARE 2905 functions to gather theprioritized digital assets for presentation according to analysis ofstatistical data recorded by activity monitor 2909.

In one embodiment, a layout fabrication engine (LFE) 2911 is provided torun on RS 2904. LFE 2911 is adapted to dynamically create or remix thelayout of a canvas based on input from asset recommendation engine 2905.In this way, a user may see the recommended assets and a preview of whatthe layout will look like using the highest prioritized assets likebackground, photos, stickers, frames for photo slots, text box styles,and other items. One component of such an engine is a machine learningroutine that learns from a library of prioritized assets includingphotos, stickers, and other prioritized assets and is adapted to studypage layouts containing these assets. The routine aids layoutfabrication by providing intelligence relative to how best to positionthose assets such as photos, for example, and how best to complimentthose assets with other assets like stickers. One example of such anoptimization is mining metadata from prioritized assets like photos tocreate relevant captions for those photos. In one embodiment, locationdata and time data might be leveraged to provide a relevant caption fora photo, for example, if a photo is taken in New York City at 12:00 PM,then a relevant caption might be “New York City at night”, or “Midnightin New York City”.

Dynamic Application of Style and Photo Effects

FIG. 38 is an architectural overview of a customer project creation andfulfillment network 3800. Network 3800 supports default and dynamicstyling of photos contributed to image and or text based projects.Network 3800 includes a Web server 3802 hosting a service Web site 3804.Server 3802 has connection to a data repository 3805 containing customerdata such as username and password combinations for logging into the Website, address information, billing information and the like.

A user operating a computing appliance 3801 has connection to network3800 and is assumed a customer of the service and registered with Website 3800 for the purpose of creating, editing and purchasing image andor text based projects like a wedding photo book, for example. Computingappliance 3801 is a desktop computer in this example; however othercomputing appliances with network access capability may be used topractice the invention without departing from the spirit and scope ofthe present invention. Computer 3801 has an initiated project 3803displayed on a computer display screen. Project 3803 is an image-basedproject containing photos uploaded to or otherwise provided to oraccessed by the system for insertion into project embedded photo slotsfor displaying the photos on a canvas style layout that istemplate-based and associated with a project theme.

The user operating computing appliance 3801 accesses project 3803, atleast in this example, by being redirected to a runtime server (RS) 3806adapted to host the graphics user interface (GUI), online editing tools,and other services to registered users. RS 3806 includes at least oneprocessor, a data repository, and a non-transitory physical medium. RS3806 has connection to a data repository 3808 containing project dataand graphics including photos for all of the projects currentlyinitiated and existing on the system. RS 3608 has connection to a datarepository 3809 containing theme-based templates created and many timesreused in projects by customer of the service.

RS 3806 has software (SW) 3807 running from the non-transitory medium onthe server. SW 3807 is adapted, in one embodiment, to create digitaltags that specify certain stylistic effects and photo-enhancing effectsthat can be applied automatically by default or automatically anddynamically to photos inserted into a theme-based template in thebackground and transparently to the user. In one embodiment, SW 3807includes a data parsing component that is able to compile and read allof the data in a project including metadata associated with contributedphotos.

When a user, such as one operating appliance 3801 selects a template fora project that is not tagged for stylistic and photo-enhancing effects,SW 3807 may, with the aid of the parser, determine the theme of thetemplate, for example, a template decorated with assets like stickersand photo slots associated with the theme “family reunion”. As the userbegins creating the project, the parser may parse titles, captions, andother data contributed to the project by the user and any others who arecollaborating with the user or helping with the project. When the userselects a photo for addition to a page, the parser gets the metadataassociated with the photo. SW 3807 uses the data to generate astyle/effect indicator or tag that can be used to tag all of the pagesand or individual photo slots in the project template. In this case, thestyle effect indicators are digital tags that specify a list ofphoto-enhancement effects and special stylistic effects that may beapplied to each photo used in the project.

Style/effect indicators may be stored after creation in data repository3810 for reuse in other templates. In one embodiment, SW 3807 tags pagesand or photo slots in theme-based templates based only on the originaldata available in the template and any data subsequently provided by auser using the template as a basis for a project. This may beaccomplished automatically and dynamically by analyzing the parsed dataagainst a set of rules that point to pre-written sets of effects toapply. Photo-enhancement effects are those effects that might be appliedto improve the way the photo looks such as brightness, contrast, hue,color saturation, etc. Stylistic effects include such special photomasking effects like sepia, antique, embossing, smoothing, mosaic,Gaussian, tile, and many other special effects.

A tag will specify one or more photo-enhancement effects, perhaps bydefault and whether or not they are necessary like removing red eye andadjusting brightness and contrast. The tag may specify the requiredeffects but not the level of application of those effects. Imagealgorithms may be provided and utilized by SW 3807 to apply thephoto-enhancing effects at the dynamically determined levels for eachphoto inserted into a project. A tag will, in some cases, specifycertain special stylistic effects akin to those effects describe above.Such stylistic effects may be pre-written effects that are appliedautomatically and by default to a photo if the photo slot or templatepage is tagged with stylistic indicators containing one or moreidentifications of the effects. While the list of effects for a tag maybe static, the level of application of those effects may be dynamicallydetermined using image algorithms that operate against a set of rules.Typically, the stylistic effects may be applied after thephoto-enhancing effects are applied so that the image algorithm has theupdated photo data to use as input for setting the level of anystylistic effects applied. Once a theme-based template is tagged forcertain stylistic indicators for each photo, it may be saved as atheme-based and tagged template in repository 3809 for reuse bycustomers.

Network 3800 includes a statistics server 3811. Server 3811 includes atleast one processor, data repository, and a non-transitory physicalmedium. Server 3811 hosts software running from the non-transitoryphysical medium of the server, the software adapted as a usage monitor3812. Usage monitor 3812, in one embodiment, is specifically adapted tomonitor customer usage of tagged project templates. Usage monitor 3812stores usage data in data repository 3813 containing template usagestatistics. In one embodiment, SW 3807 accesses template usagestatistics from server 3812 to aid in determining what types ofstylistic indicators or tags might be best for dynamic application tophotos inserted in a similar theme-based template used as the basis fora project.

The system is self learning and may refine sets of effects specified intags based on usage. For example, if users seem to prefer onetheme-based template over others based on the effects generated for thephotos, those effects may be recommended when a new template based onthe same or very similar theme is tagged. In one embodiment, the systemrelies both on data gleaned from a user selecting a theme-based templateand usage data relative to similar theme-based templates tagged in thepast to determine which stylistic indicators to use.

In one embodiment, a user may preview the automated settings for thephoto effects on any tagged template and modify or tweak those settings,including adding and deleting or activating and deactivating certaineffects. When the template is saved, it is a new tagged template. If thetag has been changed, it may also be saved as a new stylistic indicatorassociated with a certain theme of the template. Such indicators may bereused in other templates provided that they become popular with users.Indicators that exhibit rejection by users may be purged from thesystem.

In one embodiment, a single template may have more than one theme suchas first a wedding theme and then a reception theme. Likewise, there maybe a tag specifying a set of effects for all photos in the weedingportion of the project followed by a tag specifying a different set ofphoto effects for the reception portion of the project. There are manydifferent possibilities. If a user selects a template and does not likethe automated default photo effects, the user may select anothertemplate or, in some embodiments, change or modify the effects on onephoto manually and then save the changes to the remaining tags in thetemplate.

It is noted herein that the stylistic indicators may vary from page topage in a template or from photo to photo in a template withoutdeparting from the spirit and scope of the present invention. Forexample, if a customer is working on a project that uses a template withthe theme “personal tribute” or “life and times” relative to a person, apage dedicated to the early life of the person may have a stylisticindictor of gray scale or black and white for photos of the person onthat page while a page dedicated to the most recent life of the personmay have a stylistic indicator that specifies a rich color saturation ofthe photos on that page.

In one use case scenario, assume a user decides to work on a vacationbook to a tropical island. The template theme might be “Vacation” andthe theme category might be “Tropical Island”. The stylistic indicatortag for the template pages or photo slots might specify among othereffects, a specific mask or filter that gives the impression of olderpictures with cracked or worn edges. The tags can be partly relative totheme and partly relative to theme category so a user who selects atheme-based template may have different tags than another template ofthe same theme if the theme categories are different. For example, atemplate titled “Vacation” with the category “Tropical Island” may havedifferent stylistic indicators that the same template with the category“Himalayas”. Moreover there may be many different categories for asingle broader theme such as “Vacation” then categories including everycountry. The stylistic indicators may include stylistic effects thatreflect the country the user vacationed in.

Dynamic Application of Font Style and Size Effects

FIG. 39 is a block diagram depicting a fulfillment network 3900supporting automated font style and size application during remixing ofa page layout. Network 3900 may be the Internet network in oneembodiment. Network 3900 is accessed by a user operating a mobilecomputing appliance 3901, such as a smart phone or android device.Appliance 3901 has a display screen 3907 depicting a page layout 3908(a) and a remix preview page 3908 (b).

Appliance 3901 is assumed logged into the service and having connectionto runtime server (RS) 3902. RS 3902 includes at least one processor, adata repository, and a non-transitory physical medium. RS 3902 hosts SW3906 running from the non-transitory medium. SW 3906 may be integratedwith other SW applications hosted on RS 3902 that are critical to theusers' experience in creating and editing an image and text-basedproject represented by pages 3908 (a) and preview page 3908 (b). When auser remixes a page, SW 3906 records the position and other specificsize and style attributes of font associated with, text boxes, textinserts, and photo captions as displayed before a page like 3908 (a) isremixed.

RS 3902 has connection to a data repository 3903 containing all of theavailable fonts that can be used in creating an image and text-basedproject. RS 3902 has connection to a data repository 3904 containing allof the standard attributes of the available fonts in repository 3903.Standard attributes include size, referred to as “point” attributes,font colors, and font presentation styles, for example, italics, bold,cursive, etc. RS 3902 has connection to a data repository 3905containing font stylistic attributes like drop and other shadow effects,fill effects, inset, three dimensional effects, and any other specialstylistic attributes that can be attributed to fonts for printing.

When a page layout of content is remixed, often the content must berearranged and resized to fit into a new layout that may include somecontent and possible different core aspect dimensioning that that of theoriginal page. In this example, page 3908 (b) is a remix page previewthat may be shown to the user before the user commits to finality of theremix action. It is important that items caught up in the mix likephotos, text boxes, stickers, and other content be orientated andassociated correctly on the remixed page, which may also include contentif desired by a user. For example, a page with one photo may be remixedto a page that contains more than one photo, etc.

In this example, page 3908 (a) includes a photo 3909 (a) that iscentered on the page layout. Photo 3909 (a) has a caption 3913 (a). Page3908 (a) also includes a text block or box 3911 (a) containing somedescription. Page 3908 (a) further includes stickers 3910 (1-n) arrangedabout the centralized photo. The user operating appliance 3901 hasordered a page remix where the user desires the addition of a photo andassociated caption. In this example, the new layout is at the same coreaspect ration as the original layout. However, in some cases the remixedpage will be remixed according to a different core aspect ratio.

SW 3906 is integrated with SW for remixing a page and handles anychanges to font that might be made to optimize the font presentation inthe remixed page. SW 3906 records all of the positions of items on thepage before remixing and all of the current font attributes of all ofthe font on original page layout 3908 (a). In preview page 3909 (b),original photo 3909 (a) is resized to a smaller size and moved fromcenter to left upper corner of page layout 3909 (b), in part toaccommodate addition of a new photo 3912 and associated caption 3914.Caption 3913 (b) represents caption 3913 (a) modified by SW 3906 atleast in size and position to accommodate photo 3909 (b) in a new sizeand position on the remixed version of the original page layout. Photo3912 and caption 3914 are added during the remix operation. SW 3906controls the size, position and presentation of caption 3914. SW 3906has knowledge of the initial remixed position of any item associatedwith font like captions associated with photos, for example.

Text block or box 3911 (a) is modified in position and size boundary andrendered as text box 3911 (b) on remix preview page 3908. In otherchanges, sticker items 3910 are not modified in terms of size, but aremodified relative to original positions on the remixed page 3908 (b). Inaddition to modifying text box sizes, such as text box 3911 (a)(original) to 3911 (b) (size boundary modified), the font within thetext block or box may be automatically resized to work better with thenew boundary size. In original page 3908 (a), there are two fontlocations, text box 3911 (a) and caption 3913 (a). SW 3906, in general,looks at the current fonts and analyzes how current text boxes,captions, etc. currently fit around the other content on the page beforeremix. In some cases, the SW makes certain font sizes smaller or largerto accommodate other elements on the canvas. In other cases, the SWadjusts the font's colors, inset, drop shadow, or any other stylisticelements available to the fonts.

If the font styles for the font in text box 3911 (a) and in caption 3913(a) are different in the original page, SW 3906 may unify the fontstyles for both the font in the text box and font in the caption to thesame font style for presentation on page 3911 (b). SW 3906 looks at thecurrent font styles on the page before remix and can then determine aprimary font style to use across the whole page. In this way, SW 3906also controls the font attributes and presentation of the font in newcaption 3914 to be consistent with the other fonts on the preview pageto achieve uniformity.

The automated remixing and changes to existing font relieve the user ofmuch added work in manually changing font sizes and styles. Duringremixing, the system or the user may add or withhold items from or adadditional items to a preview page depending on necessity and a goal ofconsistency throughout many possible pages in a project. Uniformity isoften a good paradigm in design and aids in more cohesive page layouts.

Layer Management Across Multiple Pages

One of the aspects of building and remixing canvas pages of a project islayering where a user creates two or more layers, each layer capable ofaccepting items for display (see FIG. 10). For example, when one itemoverlaps another item, there are two layers involved, one holding thetop item and another holding the underlying item. One nomenclature fordetermining the proper layer of an object or item is a Z-index. Forinstance, if the user dragged a photo spanning two pages, to a singlepage, then back to two pages, the z-index of the photo relative to otheritems may not be its original z-index.

FIG. 40 is a process flow chart 4000 depicting steps for preservinglayer depth for an item moved between pages in a layout. At step 4001,the user is assumed in session with the server through a graphics userinterface (GUI). In this step, the user is actively working on a projectsuch as a photo book, for example. At step 4002, the user may decidewhether to move one or more items like photos from one page to anotherpage. If the user decides not to move an item at step 4002, the processresolves back to step 4001. If at step 4002, the user decides to move anitem or more than one item, the system records the original positionsincluding the Z position that indicates which layer of two or morelayers the item or items to be moved is occupying.

At step 4004, the user moves one or more items within the project to newlocations. At step 4005, the system looks up the Z data for each itemmoved. At step 4006, the system places the items in their new positionson the proper layers. In one embodiment, the user may reverse thedecision by command or edit button and the system will replace the itemsto their original positions. In this way the system always remembers thepositions of items including depth in layers. The system preserves theoriginal depth (Layer) of an item by default. However, a user may changethe layer of the item by interacting with the item such as by rightclicking on the item to change layers.

In one embodiment, the items include photos, text boxes, captions, andstickers. In one embodiment, z-index positioning of an item is preservedduring automated remixing of a page resulting in moved items. In oneembodiment, the snapshot is taken periodically during interaction with aproject page layout.

In this embodiment, the system, aided by software functions to take asnapshot of each item in a group of items arranged on a multi-layercanvas comprising at least one page. Then the system looks up thepositional data, including depth, associated with the snapshot data ofany of the group of items that will be moved and that were included inthe snapshot resulting from execution of the previous function. Finally,the system preserves the original depth position (layer) of any of theitems included in the snapshot upon movement of the item or itemsbetween pages or from a two page span to a single page.

PDF File Generation Using Vectors and Raster Data

Referring now to FIG. 30, the system employs a print engine like printengine 3007 to prepare print orders, typically in a PDF format fordistribution to a networked printer such as a third party printer 3003.Printing engine 3007 is a flash based print engine that is enhanced toproduce high resolution output files to print.

Referring now to FIG. 31, the process for printing involves steps fordetecting when a page is too large for snapshot resulting in a limiterror on the print engine process. The print engine splits the page intosmaller tiles and relies on a stitching component to reconstruct thepage for printing.

In another embodiment, the system using a print engine like engine 3007creates output files containing both rasterized data and rawun-rasterized vector data for a printer like printer 3003. The systemhas access to multiple disparate print vendors. These print vendors allaccept PDF file formats for printing, however many print vendors differin how the content is interpreted and translated into actual instructionfor the printer. Therefore, the PDFs are rasterized to make them easilyinterpretable to those print machines, more specifically by the machinesraster image processor (RIP). However, compressing raster images using aJoint Photographic Experts Group (JPEG) compression algorithm or a JPEG2000 compressor can reduce image quality.

By contrast, if a lossless compression algorithm is used such asPortable Network Graphics (PNG), then the file sizes for the final PDFfor printing can be very large for certain assets. To illustrate, for a100-page book that is 12″×12″, each page would be 3600×3600 pixels.Calculating approximately 30 megabytes (mb) for each of these pages, thetotal size for the PDF is ˜3 GB which is very large and will takeprinters extra time to download and procure these orders. Moreover,compressing images with fonts in them even the slightest bit can leave alot of undesirable artifacts which will be noticeable in print.

FIG. 41 is a process flow chart 4100 depicting steps for preparinggraphic and text assets in a project for printing. Process 4100represents a combination different compression techniques applied toselected assets coupled with other printable assets represented in theirnative vector format to reduce file size while preserving image quality.It is assumed for purposes of discussion that a user has initiated andcreated an image and or text-based project with graphics, and hastransacted with the system to procure a printed version of the project.

At step 4101, the print engine that is responsible for preparing theproject for printing receives a project for staging. A project may havea background image or layer with photos, stickers, text boxes, andcaptions. Assets may include images with or without text embedded inthem. Moreover, printable assets may occupy layers such that some assetsoverlay other assets on the canvas of the project.

At step 4102, the print engine analyzes the project and records thepositions of all of the printable assets including X, Y positioning andZ-index or layer position. At step 4103, the print engine selectscertain assets or groups of assets and analyzes each asset to determineif the asset should be compressed. For example, the software provided todetermine this may look at digital file size and other factors of theasset, such as whether there is text embedded in the asset to determinewhether the asset is suitable for some type of compression (lossless orlossy) and level or tier of such compression. If at step 4104, it isdetermined that an asset should not be compressed, the system skips theasset and the process resolves to analyze a next asset at step 4103.

If at step 4104, the system determines that an asset should becompressed, the system determines compression type and compression levelunder that type for the selected asset at step 4106. For example, if animage is large in file size and contains text, the system mightdetermine that a light lossy compression using JPEG, or JPEG 2000 mightsuffice to reduce the file size, retain the image quality, and reduceartifact and possible undesired artifacts in print. An asset may be asticker with much smaller file size and resolution than a correspondinghigh-resolution image. The system might determine a lossless compressiontype may suffice such as portable network graphics (PNG) compression.Likewise, another asset type may be a text asset or an asset that ispredominately text based. Such as asset is already thin compared to filesize and therefore may not be compressed at all but rather left invector format if need be.

The system may rasterize each compressed asset for the raster imageprocessor of the printer at step 4107. The system aided by software maythen determine at step 4108, if there were any vector assets that wereskipped at step 4105. If there were no vector assets in the project, thesystem generates a PDF with rasterized assets at step 4110. These assetsmay include several different file types in addition to JPEG and PNG.However, it is important that the file types are compatible with theprinter processes of multiple different print vendors. It is alsoimportant that the resulting overall project size is small enough thatany of the multiple print vendors can handle printing the projectwithout undesired delays or bottlenecks in printing while preserving aminimum print quality for the assets in the projects printed. Aftergenerating the rasterized PDF file for print, the process ends at step4111.

If the system determines that there were vector assets that were skippedat step 4105, the system generates a PDF with both rasterized andcompressed assets and vector assets. The process then ends at step 4111.The final PDFs are likely to contain two or more layers and vectors. Ina simplified use case, consider a project page having a background, atext box and a sticker overlaid over the text box. The filereconstruction might be a JEPG background, followed by vectorized text,followed by a PNG representing the sticker. There are many combinationspossible.

Project Data Storage Optimizations

Referring now to FIG. 25, design templates that become canvas layoutsare created in markup language from a designer tool (2502) having accessto the language known to the inventor as MBML. Fields in the structureof the template are auto populated when a project is initiated by datapreviously entered in a project wizard (2504) presented to clientsthrough the project interface (GUI).

The current structure used for storing projects couples the view withthe content. For example, currently, a page photo item entails both thepositioning data of where the item is located on the canvas as well asthe actual content (the photo). A limitation in that approach worksagainst scalability due to redundancies in replication of data and speedof access to actual content delivered to the project when it is executedfor display and editing.

FIG. 42 is a process flow chart 4200 depicting steps for optimizingstorage space and data reproduction performance relative to an image ortext-based layout with content. At step 4202, a designer accesses abasic project layout (template). There may be several different types ofbasic layouts, for example, one layout for each different project typeavailable. A basic layout for a 12″×12″ photo book will be substantiallydifferent than a layout for a 4″×6″ invitation card. All of the basictemplates may be stored for designer access.

At step 42, the designer aided by software performs data categorizationand separation for all of the actual structural data making up thelayout and content data that would be displayed within the layout andprovided by a client and by the system on behalf of the client (autopopulated content). Structural data refers to data making up thetemplate including static photo slots, title bars, content anchors, somestatic text boxes, some static caption bars, and any other mostly staticdata structures. In this step, there may be one or more degrees ofseparation relative to the structural data. A single degree might besimply to isolate all structural data and separate it from being storedwith content data. Another degree of separation might be a separation ofstructural data according to always static structural data and sometimesdynamic structural data.

At step 4203, the designer stores the structural data elements in onelocation for centralized access. At step 4204, the designer aided bysoftware links content type to appropriate data structures from thelayout. For example, certain content types are linked to the books frontcover, front inside cover, spine, back cover, inside back cover, and toa layout page. Cont type may include user-supplied content and systemsupplied content. For example, a basic layout before first use containsonly structural data and has no content. Assigning a theme to a templateadds content to the template in the form of text, background elementsand graphics such as stickers. Data added to the template fields bydefault is content that is originally supplied by the user, but retainedby the system for reuse. Dynamic content are data provided dynamicallyby the user like new photos or other graphics, new structural elementsinserted into the canvas such as an added photo slot, or new textentered into existing text fields either manually or through systeminnovation such as automatically adding captions to photos from photometadata.

At step 4205, the designer references the version of the original layoutin storage and in search engines provided to locate templates andlayouts. It is noted herein that an original layout may host multipledifferent themes without any structural changes or modifications to thetemplate. By separating the layout from the content, new layouts mightbe created only when the presentation of the content changes orstructural elements are added. For example, if a user moves an item onthe page, then the structural layout of the page is modified. Thus a newlayout may be created and saved to represent this change. However, if auser simply pans a photo in a photo slot or swaps a photo for anotherphoto, this could be categorized as a change in only the content-layerof the page. Thus, the same structural layout data can be used for allprojects where the structure of the layout was not modified.

FIG. 43 is a project flow chart 4300 depicting steps for optimizingperformance in content delivery using a template with different datalayers. At step 4301, a user opens an existing project such as a photobook, for example. At step 4302, the system aided by software gets thestructural components of the layout from a data source dedicated tostoring such components. It is assumed in this example that the projectexecuted at step 4301 has structural components that are separated indata storage from content following at least one degree of separation(structure vs. content). It is noted however that more degrees ofcomponent separation can be observed such as classifying certainstructural components in more than one class such as static components(those that cannot be changed), and dynamic structural components (thosethat can be modified by a user).

At step 4303, the system gets the project content to present in displayin the project. Content may be stored in multiple different locationsthat are accessible to the system over the network. Some photosrepresenting project content, for example, may be accessed fromcentralized content storage while others may be accessed from othersites dynamically and displayed in the project without necessitatingsystem control over the content. At step 4304, the project is displayedin the user's interface with all of the current content for editing.

At step 4305, the user edits the project. A user may edit a project bydeleting or adding content, moving content such as by remixing a pageand in some cases adding, deleting, or modifying structural componentsin the layout. At step 4306, the system determines if during editing theuser adds content. If the user ads or replaces existing content, theprocess resolves back to step 4303 where the system gets the new projectcontent and displays the content for the user at step 4304. At thispoint with no structural modifications, the new project display includespointers to all of the same structural components. The new content isadded dynamically within the content layer separated from the structureof the layout. The system may process content that is supported bystructural components like sizing photos to fit photo slots and thelike.

If the user does not modify or add new content at step 4306, the systemdetermines if the user has made any changes or modifications to one ormore structural components of the project layout. At step 4307, if theuser has made no changes to the underlying structure of the layout, thenthe process resolves back to step 4303 and the project is displayed withthe content at step 4304. The process may loop through the editing,content acquisition and display steps. If the user does not modifystructure within the structural layer of the layout, then the samestructural components are continually accessed from a single source.

If the user has modified a structural component at step 4307, the systemmay save the project with a new layout customized for the user by thefact that one or more structural components in the original layout werechanged. The new layout is stored for that user separately from theoriginal layout. The next time the user executes the project, the systempoints to the newly saved layout and does not reuse any of the originalcomponents. In another embodiment where structural components areseparated from one another by class, versions of these components mightbe created. When the user executes the project at next opportunity, thesystem will still get all the original components that have not changed(version 0) and it will access any modified components having a newversion number (version 1). In one embodiment, the new components thatwere modified are written to storage in the form of a new addressed datasource mapped to or indexed to the parent data source reflecting eithercontent data changes to the image-based layout, structural data changesto the image-based layout, or both.

In one embodiment, all templates have a set of basic components that arestatic components (read only) and may not be deleted or modified. Inthis regard, all users having projects that depend on the same originaltemplate are pointed to the basic original layout (structural componentsonly) to these components need not be duplicated in storage. Customstructural components like added photo slots, modified text boxes, etc.may be written to storage separately from the original layout andversioned so the system may access the same components for the user'sevolving projects.

In some cases, users may create a new component that can now be reusedby other users making identical modification requests. The system isable to determine the exact nature of the modification of one or morestructural components of a layout by a user and can perform a lookup tosee if there are any components stored for reuse that match exactly oreven closely to the modification made by the user. In preview mode, thesystem may aggregate data and display one or more suggested componentsthat were modified by other users and saved for reuse based on anyparticular user request. If the user accepts a previously modifiedcomponent then that component is used and the version is saved for thatuser pointing the system to the new resource the next time the userexecutes the project using the same component.

In one embodiment, data separation in a layout to separate structurefrom content to a first degree may be accomplished using a markuplanguage such as MBML described further above and content presentationmarkup language similar in some respects to cascading style sheet (CSS)script used to separate content from structure in hypertext markuplanguage (HTML).

Enhancing Color Options for Content

Referring now back to FIG. 10, the system provides a color readingmodule or color analyzing tool 1008 for users to read the colors ofassets displayed in their projects. This tool may also be used to applya color to an asset, save a color, and blend colors to create newcolors. Certain assets provided to a layout as colored assets likephotos, stickers, and the like may have very different color attributeslike color saturation, color range, etc. One goal of the presentinvention is to enhance the overall experience of editing colors inassets individually and in groups.

FIG. 44 is an architectural overview of a network 4400 supportingenhanced color options for displayed content. Network 4400, is theInternet network in this example. However, network 4400 may be acorporate wide-area-network (WAN), local area network (LAN), or theInternet and any connected sub-networks. Network 4400 supports a runtimeserver (RS) 4401. RS 4401 includes at least one processor and a datarepository, the server including a non-transitory physical medium.Server 4400 hosts software (SW) 4408. SW 4408 includes all of thecomponents required to enable active editing of image and or text-basedprojects. SW 4408 also includes components for enhancing color optionsfor both the system and user, including provision of searchfunctionality for matching colors and contributing new colors to aproject asset.

Server 4401 has connection to a repository 4407 adapted to contain allof the structural data relative to users' existing projects includingtemplate data. Server 4400 has connection to a repository 4405.Repository 4405 is adapted to contain all of the existing assets fordisplay in users' projects. Assets include photos, stickers, text, themecomponents, and the like. RS 4401 has connection to a repository 4404and to a repository 4406. Repository 4404 is adapted to contain alibrary of words, which may correlate to certain colors. Repository44065 is adapted to contain a library of numerous colors. Separatephysical structures for the repositories are meant only for discussionpurposes in separating function. In one embodiment, all of the dataclasses might be stored in a same data location. Moreover, such data maybe stored and served from a data cloud or a content delivery network(CDN).

Network 4400 supports a client depicted herein as a desktop computingappliance 4402. Appliance 4402 will hereinafter be referred to in thisspecification as computer 4402. Computer 4402 is presumably operated bya user and has, in a state of display, a current project 4403 the useris in the process of building and or editing. Therefore it is assumedherein the user operating computer 4402 has connection to RS 4401 overnetwork 4400. Project 4403 may be a photo book and contains severalassets that are typically rendered in color (unless black and white) onthe display. In this example, the assets include a photo 4409, a photo4410, and stickers such as sticker 4412. Project 4403 may also includebackground assets, text boxes, caption boxes and so on.

SW 4408 performs, among other tasks, color analysis of assets relativeto projects stored on the system. In a preferred embodiment the functionanalyzes color in at least three dimensional values to determine all ofthe colors of an asset presented in image-based project 4403. SW 4408creates a histogram for each asset analyzed based on the findings of thecolor analyzer component. SW 4408 may augment the histogram by addingnew color contributions to the histogram enabling enhanced color controloptions for the asset. Color contributions may be added by users or bythe system without departing from the spirit and scope of the presentinvention. Once a histogram is created for an asset, each time the assetis stored the histogram is also stored with that asset.

In one embodiment, the color analyzing component of SW 4408 utilizes a120 color scheme from high performance photo gallery (HPPG)color_palette.txt. In another embodiment, the analyzer relies on namemodule and color_hex_names.yml (140 colors from w3 supported colorlist). In a preferred embodiment, SW 4408 enables a color searchcovering hue saturation and luminance (HSL) using histogram-basedindexing to look up relevant data. In a preferred embodiment, the systemuses “color distances” in Commission Internationale de I'Eclairage (CIE)lab or (CIELAB) as known to the inventor to calculate the relevancies ofcolors.

For example, to round a color, say a color C, the SW takes the color Cand compares it to each of the colors in the color scheme or palette.The closest color via cie00 distance is the top color returned for the Cround. Neighboring colors are those that are closest to one another,having a relatively small cie00 distance. Names of colors like red orlight blue, for example, are stored on disk in color_hex_names.yml.Colors may also be matched to words, in one embodiment, and word can beassigned or tagged to colors.

In one embodiment, a process is provided for checking if a word (W) is avalid color in the system. ColorAnalyzer::Name extendsCacheContentstore::Memcache and is a hash of [“color_name”=>color_hex],for instance W is submitted into a search function find_by_name in thesystem to determine whether the word is a valid color or not. If W isvalid, a hex is returned. If W is invalid, nil is returned. The hexcodeof a color is the hexadecimal representation of the colors red greenblue (RGB) values. In the process, relevance of one color to a coloredasset is a float between 0.0 and 1.0. The float value represents theproportion of which the asset, like an image for example, is comprisedof a given color. A float value of 1.0 means the asset is all thatcolor. A float value of 0.5 means the color comprises 50 percent of theasset color. A float value of 0.0 means the color is not found in theasset.

The histogram generated for each asset is a hash of [color_hex=>thanrelevance] for that asset and represents the distribution of colors inthe asset. In one embodiment, the system is able to facilitate blendingof colors. In this embodiment, SW 4408 leverages a fast (open sourced)word search platform called SOLR pronounced soh-lar created by Apache™.For example, if an asset like an image A is 75 percent “red”, the querymay ask for “dark red”, “light red”, or even “orange”. Though thesecolors do not match red, it is desired that they are returned in thesearch results, albeit with a lower score that “red”.

SW 4408 gets the histogram for “red” and sees that {“red”=>0.75. The SWthen reads all the other colors in the histogram. For each color foundin the histogram, the SW calculates all the neighboring colors and theirrespective CIELAB from the target color “red”. Orange is 5 away from redwhile light red is 2 away. Therefore outputs for red-similar colors[[“red”, 0], [“lightred”, 2], [“orange”, 5]]. Each of these data areincluded in with the relevance of the original histogram, for example,[lightred”, 2, 0, 0.75]. Therefore each of the entries represents anadded color to the new “blended” histogram. A generalized formula,contribution=original relevance/distance. For example, a combination of“lightred”=0.75/2.0. SW 4408 adds up the contributions in a hash togenerate the new blended histogram. It is noted herein that ablended_histogram may have values (blended/relevancies) exceeding 1.0.

In one embodiment, the system uses a library, analogous to word_lib4404, of thesaurus terms broken up by nouns and verbs. These words areused to expand queries to return additional color results for users. Thepreferred format for the search platform is java script object notation(JSON), however, other formats or languages may be used instead of JSONwithout departing from the spirit and scope of the present invention. Anexample entry might look like<pre>{“yelp”:{“verb”:“squeal”,“shriek”,“howl”,“yowl”,“yell”,“cry”,“shout”,“bark”,“bay”,“woof”,“holler”}}</pre>. These terms occupy one line or string inthe data file.

Parts of speech that might provide utility include nouns, verbs,adjectives, prepositions, adverbs, exclamations, pronouns, conjunctions,in plural and cardinal forms. The values corresponding to each keywordor part of speech comprise nested arrays. The values returned may alsobe augmented using a multiplier in the original string of words. Forexample, <pre>{“words”:{“part_of_speech:”related word1”, “relatedword2”,: 0.5”, “related word3: 1.5”}</pre>. In this example, relatedword 2 has a multiplier of 0.05, therefore it will have half of therelevancy of a normal related word. Related word 3 is multiplied by 1.5so its relevance will be increased by a factor of 50 percent. Words thatare not multiplied have a 1.0 relevancy by default. In this way,relevancies of related words may be adjusted from the baseline enablingtweaking of the system for very popular search terms.

SOLR and Sunspot, a component that enables searching through SOLR, areboth leveraged in one embodiment of the present invention. SOLR uses anindex data structure for search functionality. A represented asset inthe index is termed a document and is associated or stored with its ownblended or original histogram. SOLR runs searches against the terms asdescribed above. Example terms might include color names and other words(nouns or verbs) like “red”, “dog”, “blue”, and or “car”. SOLR scoreseach document using a formula involving the summation of a document'sterm_scores. At the end, the results are ranked by highest score withscores of zero being omitted from result set.

A term_score is the contribution to a documents (asset) score from thematch of a given term in the document. Payloads are custom fields withinthe index which can vary from document to document. They allow for aspecific document to augment the score for a specific term match, whichmay not happen for other documents matching the same term. A payloadentry comprises three elements; the term itself, a separator, and thepayload value, for example “red”/1.0. Payloads are used in browse_assetsto supply a relevance value for a given color in an image so it can bescored later on. A sunspot query is a user-supplied string that sunspotformats then hands off to SOLR.

The system of the invention leverages a couple of other SW tools, namelyImageMagick and swrender which as off the shelf tools known to theinventor. For an asset such as a raw image, the system aided by SW 4408generates a histogram by detecting the file extension of the asset, forexample.swf. The preferred format for an asset is .png. Therefore, theasset is converted to .png using swrender. A temporary file is createdfrom the .png asset for working with the asset without directlyaffecting the original asset. The alpha channel of the asset is removed(opacity layer) using ImageMagick and color reduction is performed toreduce the colors in the asset down to 50 colors.

Each of the 50 colors are “rounded” as described further above to theclosest respective colors in the system color scheme that includes atleast 120 colors. Rounding produces a hash of the color and pixel countof the asset. The hashes for each color are normalized to turn the pixelcounts into percentages of color for the entire asset. All coloredassets in the database have a histogram including a color list expressedas color_list in code. For each key value (Kv) in the stored histogramfor an asset a tag is created in the color list. The tag value can beexpressed as tag.value=color,tag.relevance=normalized_proportion_of_color_in_image.

Blending relevance refers to both color and term tags. The contributionof related “colors” or “terms/words” for a particular asset iscalculated. The blended relevancies are calculated during index-time forSOLR, into a “payload’ field. The contribution score consist ofcontribution=t.relevance/distance where t is tag terms or tag colors.For colors, a histogram is used to derive the relevancies for the assetsand distance. In the case of terms, the particular order in whichrelated words were parsed from a dictionary source determines the“distance” away from the term. Blending is then the sum of allcontributions>=0.01, which is the value (or weight in SOLR) for the“payload” field for a given “tag”.

The query syntax for a search query uses the special operators ORdenoted internally by — and default joining operator AND denotedinternally by*. For example, red car=>return documents which match bothred and car red OR car=>return either documents which match red or car(higher score if both match since sub scores are added). Red bluecar=>returns documents which match red and car and blue red OR bluecar=>returns documents which match red and car or blue and car. This ismore efficient than running two searches, one for red car and the otherfor blue car. Red blue OR car=>returns documents which match red andblue or red and car. Fields include attributes like tag_count, width,etc. For example, [width:15 height>10 usage_count<2],[color_count.between(10,20) file_ext.any_of(jpg, gif, png)basis_template_ids.all_of(1,2,3). Search terms can be boosted inrelevancy by multiplier, for example, grasŝ5.0 car̂22.0. Boost terms areset to 1.0 by default. In scoring a document, the formula used isdocument.score=&sum;(matching_term.boost * payload_tags.payload).

The system of the invention includes several contexts in which tags maybe added. The contexts may include styles, categories, colors, and tags.Color tags are predetermined before entering them into the system. Colortags are therefore not edited by users. Styles and categories are alsopredefined in list form. All other tags or labels will go into the tagcontext, which is open-ended. In a preferred embodiment, tags may alsohave relevance to an asset. When adding tags, the relevance of that tagto that asset may optionally be input as well, on a scale, in oneembodiment from 1-10. If the relevance is not added, it defaults to 10in this embodiment. If a tag contains multiple words like “New York”,for example, it is stored in the database as “New York”. When that tagis indexed, additional tags of “New”, “York” are added to “New York” toaccount for all possible search possibilities.

Assets (HostedImageAsset) in the system may have a lifetime from 0 tosearchable. Given the asset data can reside locally or at anothernetwork source such as a hypertext transport protocol (http) source, wecan reanalyze the color tags (#reanalyze_color_tags) on the assets toget the colors into an indexable state. For example, when an asset iscreated, the after_create hook is triggered and puts the assetidentifier (id) in the analysis_queue and index_queue for the class ofthe asset. A scheduled rake task then flushes the analysis queue andcalls a routine analyze_color_tags! for every tag in the queue. If anerror is raised relative to an asset during the process, the asset isput on an error list (analyze_error_list) to be reanalyzed later.

The routine analyze_color_tags! calls the color analyzer component toretrieve a new histogram hash from the asset raw data. In this routine{palette_color is=>relevance} where palette color is 1 of the at least120 colors, and 0<relevance<1. The histogram generated is then passed toTagtical or some similar tool to create Tag::AssetColor for each ofthose colors with the relevance values provided in the hash. All ofthese tags are saved to the database.

In one embodiment, an administrative interface is provided and adaptedto be used to check whether or not a color analysis was acceptable andto enable manual term tagging of assets. In manual term tagging, theasset is examined using the administration interface and theadministrator uses the interface to create a string of term values andtheir relevancies. For example, if an asset is a picture of a tree, theadministrator may just type in “Tree” and then hit enter. If theadministrator wants to control relevancy with a multiplier, theadministrator may enter, for example, ‘tree:5’ for a tag relevance tothe asset of 50 percent. Relevancies are stored according to a floatfrom 0 to 1, but when displayed, the data is normalized to from 0 to 10for the user. When the user types in and enters the tag string, thesystem recognizes whether the tag is a category tag or a style tag andwill display it in the appropriate category or style subclass section ofthe tag.

In one embodiment during tag indexing, a routine termedActiveRecordMixins::Indexer gives the tag the method reindex!. Ascheduled rake task flushes the index_queue calling reindex! on batchesof the index_queue. If batch fails to reindex!, then the entire batch isplaced on index_error_list to be re-indexed later. Within the indexer,it is specified that the asset is searchable based on all of the termsand colors specified previously for that asset. The routine #payload_tagconverts it into a useable string for SOLR search functionality. Theroutine #payload_tag calls a routine #tags_to_hash for all tag types(color, term, style, category, etc.) of a given asset such as a sticker,for example. #tags_to_hash analyzes all tags applied to an assetaccording to all specified tag types or classes. In one embodiment foreach analyzed tag, a routing similar_values returns a collection ofvalues that are similar to the original tag value along with theirrespective “distances” from that tag value. In one embodiment, thedistance values are normalized to 100 so that the distances returned bythe similar_terms routine emulate the similar_colors distance scoring inCIELAB.

In a preferred embodiment, the platform includes various and sundrysearch filters that can be used alone or in conjunction with each otherby employing OR and AND operators. Search filters may include a fileextension (file_ext) filter for returning assets of specified fileextensions such as file_ext:jpg. A filter for width and heightdimensions may be provided that can be used with the < or > operators,for example, width>400. In one embodiment a usage count (usage_count)filter is provided to search for an asset based on the number of timesit has been used in all projects in the system. The operators < and >can be used as well as a definitive number, for example, usage_count:10or usage_count>10. Another filter might be one adapted to search for anasset based on a count of attributes such as color_count<10. Thisapplies to asset classes, style color, category, and tag.

Simulating Motion Dynamics for Digital Assets Based on User Input

Referring now back to FIG. 29, the system, in some embodiments, includesa physics simulation engine 2907 for simulating motion dynamics foritems placed in a canvas layout.

Referring now to FIG. 34, motion dynamics are simulated for digitalitems 3407, 3408, 3412, and 3413 based on preconfigured interactiveboundaries attributed to each item. When pre-assigned physicalboundaries of the items on the canvas intersect as a result of one itembeing moved in close proximity of another item, motion dynamics aretriggered according to pre-assigned physics of the boundaries, forexample one object repelling another object.

In one embodiment, motion dynamics may be triggered for a digital itemor asset placed on canvas based on physics properties assigned to theasset and the nature of user input relative to interaction with thedigital asset. In this embodiment, the motion dynamics may occurindependently of the proximity of other objects. In another embodiment,the motion dynamics may occur in conjunction with different motiondynamics triggered by boundary intersection between two or more items.

FIG. 45 is a process flow chart 4500 depicting steps for applying motiondynamics for digital assets. At step 4501, the system accesses one ormore digital assets. The digital assets might be photos, theme graphics,photo slots, text boxes, stickers, or any other digital items that maybe added to the canvas of an image and or text-based project. There arenumerous ways in which the system may access or obtain such digitalitems.

In one embodiment, the system accesses the items from local or remoterepositories adapted to host or store such assets. In one embodiment,the system obtains the digital assets directly from a user via digitalupload of the assets by the user to the system server. Typically, thismight occur while the user is building a project and uploading items foruse in the project. In another embodiment, the system accesses theassets from a remote network-connected source such as a user's socialinteraction account.

At step 4502, the system assigns one or more pre-specified physicsproperties to the digital asset or assets. Pre-assigned physicsproperties are designed to facilitate free-flowing movement of thedigital assets based on the nature of user input in interacting with theassets. In one embodiment the physics properties are pre-written by asystem knowledge worker or administrator. In this case, the system mayapply such attributes to assets in an automated and dynamic fashion asassets are analyzed. Assets may be accessed cardinally, or in batcheswithout departing from the spirit and scope of the present invention. Inone embodiment, the system server leverages a physics engine to rendermotion dynamics for certain digital assets. In another embodimentcomputer animation techniques are leveraged to emulate the function of aphysics engine for animating certain assets.

At step 4503, a knowledge worker or system administrator defines userinputs for triggering simulation of specific motions according to thepre-assigned physics properties of specified assets. The attributesincluding types of motions and types of inputs that trigger simulationof motion for assets may be stored with the assets or stored in alocation apart from the assets but tagged to the assets. In step 4504,the system stores the assets configured for motion simulation for lateruse. In one embodiment, the assets are public assets that are availableto all users. In one embodiment, the assets are private assets availableonly to certain users authorized to use them. Interactive assets thatare publicly available may be triggered for motion by any activelyconnected users that might be working with those assets in a project.Interactive assets that are private assets may be triggered for motionby users whom are authorized to use those assets in one or moreprojects.

It is noted herein that in one embodiment, the assets are prepared formotion simulation manually and such assets exhibit motion uponsubsequent interaction with the asset by a user based on thesystem-recognized type of input. In one embodiment, motion dynamics andinput triggers are pre-defined by the system before accessing anyassets. In one embodiment, the system may dynamically prepare an assetfor motion dynamics based on some indication from the user or from thesystem that the particular asset should exhibit some type of motion uponcertain types of input actions.

Input actions may include traditional inputs like mouse movements suchas drag and drop, which represent inputs from a single point of control(mouse, trackball, mouse pad, etc.). Input actions may also includethose that are possible on a touch screen or touch pad such as sweepingmotions, expanding motions, reducing motions, and other input actionsthat are representative of inputs from multiple points of control (touchscreen, touch pad, etc.).

At step 4506, a user executes a project associated with interactiveassets. At step 4507, the system loads the project with the interactiveassets. Loaded assets may remain motionless until triggered by a systemrecognized form of user input in interaction with one or more of thoseinteractive assets. While the project is displayed, the system monitorsthe user or users for input relative to interaction with one or more ofthe interactive assets at step 4508. Interactive assets may includethose assets preconfigured for interactive boundaries and thosepreconfigured for simulated motions.

At step 4509, the system determines if a user input recognized as amotion triggering input relative to an interactive asset has occurred.If no user input for triggering motion simulation of an asset isdetected, the process resolves back to step 4508 where monitoringcontinues. If the system has detected a user input recognized as atrigger for simulated motion for one or more assets at step 4509, thesystem determines the nature of the input at step 4510. The systemsimulates the motion physics of the interactive asset at step 4511 basedon the determination made at step 4510.

In one embodiment, the system detects a subsequent selection of theasset for positional modification on the layout from the perspective ofat least one predetermined aspect of user input control, for example,selecting and moving an asset using a mouse operation. The predeterminedmotion might be an inertia property where the asset continues to slideacross the canvas when the user drops the item during the positionalmodification. In another embodiment, a user may expand the size of theasset using a touch screen input such as one that zooms in content usingmultiple fingers. The predetermined motion might be to slowly expand theobject until the user taps out to stop the expanding motion. In anotherembodiment using multiple point of control, the user may make a sweepingmotion across multiple interactive assets to sweep them around and havethem bump into one another or partially overlap one another.

In one embodiment there may be a layer assignment property attributed toan asset such as this asset is always in the top layer. A second assetmay be assigned always to a next layer down such that a user may slidethe asset in the second layer to have it slide under the asset assignedto the top layer only. In one embodiment using multiple points ofcontrol such as multiple fingers on a touch screen, a user mightsimultaneously slide multiple objects around on the canvas using afinger for each object. In a variation of this embodiment, a user mightmove, rotate, or expand an asset using a combination of fingersrecognized as a specific touch screen input. In still a furtherembodiment, certain hand gestures might access different behaviors inthe user interface such as panning, zooming, sliding, bouncing,rotating, flipping, and otherwise moving assets. In one embodiment,assets may be configured with interactive repulsive boundaries andmotion dynamics so that a user may propel one asset into another or agroup of assets and what secondary motion dynamics for the other assetstriggered by boundary interactions where all of those assets occupy thesame canvas layer. There are many possibilities.

Auto-Recognition of New Photos

In one embodiment of the invention, the system can access client assetsthat are stored at remote network locations on behalf of the client.

FIG. 46 is an architectural overview of a fulfillment network 4600 thatsupports automatic recognition of and captioning of photos contributedby clients building or editing an image and or text-based projectthrough a graphics user interface (GUI). Network 4600, which is theInternet network in one embodiment, includes an access network 4602.Access network 4602 may be any digital sub-network having access andconnection to network 4600, also denoted herein by network backbone4604. Backbone 4604 represents all of the lines, equipment, and accesspoints that make up the Internet network as a whole.

Access network 4602 supports network access for users operatingcomputing appliances 4605 (1-n). Users 4605 (1-n) are assumed connected,in this embodiment, to a runtime server 4603. In one embodiment, theusers are redirected to RS 4603 via a service Website (not illustrated).RS 4603 hosts a graphics user interface (GUI) adapted to facilitatebuilding, editing, and purchasing image and or text-based projects suchas a photo book, for example. Instances of GUI 4610 are displayed oneach of the user's devices further illustrating network connectivitywith the server.

RS 4603 hosts software 4611 executed from a non-transitory medium on theserver. SW 4611 is adapted to access social interaction accounts andphoto site accounts of authorizing members for the purpose of collectingdata including photo or image data about friends and family of thosemembers, whose photos may be hosted on the network and accessible to themembers. Backbone 4604 supports social interaction and photo sitesrepresented herein by network servers 4612, 4614, and 4617. Each serverhas connection to at least one data repository, repository 4613 forserver 4612, repository 4615 for server 4614, and repository 4618 forserver 4617. Repositories 4613, 4615, and 4618 are adapted to containimages that are hosted by such social interaction and photo siteservices patronized by users 4605 (1-n).

In the system of the invention, users that are in session and working onone or more projects may upload images or photos to those projects fromtheir devices. In other embodiments, the user may tag or otherwisespecify and authorize the system to get images or photos from anyservice site the user specifies and where the images or photos arehosted on the network. Therefore in a project display, there may bephotos uploaded by the user and photos accessed from elsewhere on thenetwork displayed in the project viewable through the GUI. In thisexample, user assets including project photos or images are stored indata repository 4609, which is connected to RS 4603.

SW 4611 includes an image-based facial recognition system encapsulatedin an automated browser instance capable of navigating to a siteuniversal resource locator (URL) and logging into a social interactionaccount or a photo sharing or hosting account on behalf of the user. SW4611 is trained to perform the functions mentioned above using dataprovided by the members and under sanction by the members to accesstheir sites. In one embodiment, members may have photos hosted at apublic site or elsewhere on the network where no authentication isrequired for access. However, many of these sites are password protectedrequiring system access to the user's passwords and other requiredauthentication data for those sites. SW 4611 may operate on behalf of auser whether or not that user is online. SW 4611 accesses user images offriends and family and evaluates them for facial recognitionpossibilities. Of all photos that might be posted in a user account,many will not be of interest to the system due to a lack of discernablefacial features that might be leveraged to “recognize the same or asimilar photo added to a user's project.

In a preferred embodiment and for each authorizing user, SW 4611navigates, during session and or in the background for offline users,with the aid of browser functionality to one or all of servers 4612,4615, and 4618. For each user, SW 4611 browses the photos hosted on theserver such as the photos stored in repositories 4613, 4615, and 4618.SW 4611 is adapted with the aid of a facial recognition component todetermine if a photo or image being looked at has a recognizable face init or not. If not, for example, the photo may be of a dog or of a cat,or of scenery, a building, or any other thing other than an image havingat least one recognizable face of a family member or friend. When SW4611 comes across a photo that has at least one face, SW 4611 accessesthe image for further processing. A facial recognition pattern iscreated for each qualifying photo or image. Likewise, SW 4611 collectsany descriptive photo data (metadata, data) about a qualifying photo.

Facial recognition patterns and associated data/metadata are stored in adata repository 4607 connected to a storage server 4606 in a datastorage cloud 4601. It is noted herein that it is not required that databe stored in a data storage cloud or island in order to practice thepresent invention. Distributed computing and data storage offeredthrough cloud services is a convenient and readily scalable storage andaccess solution. Users 4605 (1-n) typically contribute photos to theirown projects and often to projects initiated by others. Photoscontributed to projects are illustrated in data repository 4609. GUI4610 is integrated with a monitor component that detects images orphotos uploaded to the system by users. Typically, the photos areuploaded and then placed within the user's project at the control anddirection of the user or on behalf of the user such as in a remix.

SW 4611 is further adapted to monitor user activity relative to photocontribution and may intercept any photo contributions the user makes.SW 4611 intercepts such photo contributions while the user is in sessionand has just made the contribution(s). SW 4611 first determines if acontributed photo qualifies for facial recognition. If so, SW 4611generates a facial recognition pattern of at least one face in thephoto. SW 4611 may also collect any photo data and metadata such asadded description or caption text attributed to the photo and any tagdata associated with the photo. For example, a photo containing afrontal rendering of a person should have face data visible to SW 4611,which may recognize basic facial features to first qualify the photo forprocessing. Tag data, such as when a friend tags another friend in aphoto is also collected for the name data.

SW 4611 attempt to match facial recognition patterns taken from newphoto contributions against facial recognition patterns stored inrepository 4607. If a sufficient match is made between a facialrecognition pattern taken from a new contribution and a facialrecognition pattern stored in repository 4607, the SW may then create acaption and or description from the metadata and data stored with theoriginal facial recognition pattern. Therefore, the likelihood that thecontributed photo is the same individual whose photo was earlieranalyzed, can be controlled by threshold (percentage of matchingfeatures in the pattern) and can be further validated by matchingmetadata and data from the contribution to data and metadata takenearlier and stored in repository 4607.

In this example, the system may “suggest” a caption, name tag, or otherdescriptive indicator that includes a name for the person or persons inthe photo contributed to the project by sending notification to the userthrough the GUI that includes such recommendations or suggestions forcaption or other description for the photo in the project. The usermight accept or reject the recommendation. If the system is correct, theuser may save time by adopting the system recommendation for captioningetc. If the system is not correct, the user may disregard thenotification and recommendation. In one embodiment, the contributedphotos are auto-tagged with name data and or other data based on a matchand the availability of the earlier data/metadata.

If a user Jim has a single photo of a friend Jane on Facebook™ and Janehas a sister Joan that Jim happens to have a single photo of on anothersite, for example, the single photos of Jane and Joan are pre-analyzedfor facial recognition and any associated data/metadata such as thenames Jane and Joan. Later, the user uploads a photo containing afrontal view of two people Jane and her sister Joan. That photo may beintercepted and analyzed for facial recognition patterns to use thosepatterns to see if they match any patterns previously generated andstored for the user. In this case, the system would take the facialrecognition patterns of both Jane and Joan, using them as input tosearch against all of the patterns taken from photos at one or moresites of the user.

The system would identify both Jane and Joan in the project contributionphoto, and may have some metadata that reveals the relationship betweenthem. The suggested caption could be “Jane and her sister Joan” or “Joanand her sister Jane”. The system may have data indicating the importanceof both to the user where Jane is dominant in Jim's online life lendingto the first recommendation. If there is any photo data or metadataalready associated with the user's contribution like place name, time,date, or the like, the caption or suggested description for thecontribution may contain the added information such as “Jane and sisterJoan at Dune State Park in July”. Therefore, using a combination offacial recognition and associated descriptive data, the system canmanage identification of more than one person in a photo contribution.

FIG. 47 is a process flow chart 4700 depicting steps for matching facialrecognition patterns in photos to pre-identify photo contributions to aproject. At step 4701, the system aided by SW, accesses socialinteraction (SI) sites and photo hosting sites patronized by the clientor user. In this process, the SW requires user authenticationinformation and permission to access the photos on behalf of the user.The information provided by the user is leveraged in an automated loginprocedure to gain access to user sites. In this step, the SW may accessthe user's profile photos, posted photos on the user's socialinteraction feed or wall, and any photos previously uploaded by the userto photo hosting sites like Facebook, or Flickr™

At step 4702, the system browses client photos looking for candidatephotos that include one or more face shots that are of sufficientquality to generate a facial recognition pattern for one or more facesin the photo. Photos that do not qualify as a face photo are skipped. Atstep 4703, the system determines if an accessed photo is a candidate forfacial recognition. If a photo is not a candidate for facial recognitionat step 4703, the process resolves back to step 4702 where the systemcontinues to browse user photos. If a photo is determined to be acandidate at step 4703, the SW generates a facial recognition patternand collects any photo data and metadata that may already be associatedwith the photo. If the photo shows a single person's face and dataassociated with the photo includes a name, the system may infer that thename is that of the person in the photo. If the photo metadata includesa place name, then the system may infer that the place in the photo isthe named place. If there are two faces in a single photo, the systemmay attempt to generate facial recognition patterns for each facedocumented.

At step 4705 for each candidate photo processed, the system stores thefacial recognition patterns generated from photos and the associateddata and metadata associated with the photos in a connected repositoryor other storage location that is accessible to the server and that maybe searched using pattern matching and keyword matching. The data thatis stored is attributed to the client by identification (ID) such asclient name, number, password, etc. In a preferred embodiment, thephotos themselves are not taken or copied from user sites. In oneembodiment, the system may keep a thumbnail of each processed photo forvisual identification purposes. Other data about photos collected mayinclude file information like photo resolution, size, date created, andimage number like (DSC11287), for example. Depending on the location ofthe photos including the venue of posting, the data associated with thephotos may be minimal or very rich, for example, a simple caption versesa paragraph of description for a photo.

Steps 4701 through 4705 may be taken by the system in the backgroundwhether a user is online or not online. The process may be undertakenperiodically, by schedule, or by direction from a user. Data processedon behalf of a client are stored with client ID and are not used for anyother non-authorized purposes. Older data or obsolete data may bepurged.

At step 4706, a client may log into the Website hosting the photoservices. At step 4707, the client executes a project for editing, orinitiates a new project for build. At step 4708, the system monitorsuser activity during session looking for new photos added or in theprocess of being added to the project. A step 4709, the systemdetermines if a new photo is added to the project. A new photo might beone not previously documented by or used in the system. If no new photosare detected at step 4709, the process loops back to step 4708 forcontinued monitoring. If the system notices that the user is introducingone or more photos to the project at step 4709, the system determinesfor each photo if that photo is a candidate for facial recognition. Ifthe system determines that there is no new photo at step 4710, theprocess loops back to step 4708.

At step 4709, if the system determines a photo is a new photo, thesystem checks at step 4710 whether the new photo is a candidate forfacial recognition. This step is analogous to step 4703 for remotephotos and photos already in the system. If the photo is not a candidatefor facial recognition, then the process resolves back to step 4708. Ifthe new photo is a candidate for facial recognition at step 4710, thenthe system aided by SW generates at least one facial recognition patternand collects data and metadata associated with the new photo at step4711. In one aspect, the photo may not have rich data associated with itif it has not yet been posted anywhere on the network. However, if thephoto is a remote photo that the user wants to include in the projectwithout direct submission, then the photo may have rich data associatedwith it.

Step 4711 occurs while the user is uploading the new photo or after theuser has submitted the location of the photo for use in the project. Atstep 4712, the system conducts a search against the stored facialrecognition patterns and data taken previously from photos the user haspermitted access to. The system uses the facial recognition pattern ofthe photo generated in step 4711 as primary search criteria. Any data ormetadata associated with the photo may also be used in the search,possibly to match data and metadata associated with a previously storedfacial recognition pattern.

At step 4713, the system determines if a significant match was found inthe search of step 4712. A significant match includes matching facialrecognition patterns to a satisfactory degree since if it is not theexact same photo, the patterns might differ somewhat. If the systemfinds no match at step 4713, the process resolves back to step 4708where the activity of the user is continually monitored during thesession. If the system determines a match at step 4713, then the systemextracts at least a name from the stored data at step 4714 to attempt toidentify the subject of the matching facial recognition patterns. Thesubjects name and other data might be included in the data associatedwith the matching facial recognition pattern.

Notwithstanding minor pattern variations that may occur between two faceshots of a same person, matching data may help the system to betterconfirm identity at least of the person subject to pattern recognition.For example, two patterns are close but do not match exactly where thename data for both sets of data collected is the same. This may occurwith to separate photos of the same subject. If the new photointercepted at step 4711 were the exact same photo previously stored,the patterns would match identically, perhaps within a very small systemerror margin.

At step 4715, the system sends a recommendation to the client for aphoto caption or for a photo description. The recommendation stems fromthe data and metadata associated with the stored facial recognitionpattern. The user may accept the recommendation or not in oneembodiment. In one embodiment, the system auto-populates caption ordescriptive fields and the user may change or edit the fields ifdesired. The caption or description might include the names of one ormore people in the photo, the place in the photo, the data the photo wastaken, and other data if available. At step 4716, the system determinesif the session is over. If the session is or has ended at step 4716, theprocess ends for that client at step 4717. If the system determines thatthe session is ongoing, the process resolves back to step 4708 foractivity monitoring.

It is noted herein that a subject (person) in a photo contributed to aproject might be known to the client and to family and friends of theclient. Therefore, pre-stored data may be made available for searchpurposes to more than one client if those clients are friends outside ofthe system and, perhaps within the system such as collaborators, etc. Inthis way a pool of available photos and information might be shared byseveral clients without departing from the spirit and scope of thepresent invention.

In a further aspect, a client may have a pool of photos that are taggedand already used in projects. These photos may be processed as describedearlier for facial recognition pattern and data/metadata where theinformation is stored as described at step 4705. However, instead ofdoing a search to obtain identification of a person or persons in a newcontribution, the search may be conducted for any matching photos thatcould be posted at several different locations accessible to the user.The photos returned become a pool of possible photos the user may browsethrough to select photos to add a project. In this embodiment, theactual photos may be retrieved or simply the Universal ResourceIndicators (URIs) of the photos with thumbnails of sufficient qualitythat the user can access them.

Managing Snap Features

Snap features refer to canvas grids configured for snap-to so that itemsplaced on the grid align to one another more easily. Digital assets likephoto slots, text boxes, stickers, and other digital graphics are, incertain instances, configured with snap guides that snap the item to agrid line or anchor. The inventor provides a snap-to prioritizationmethod that aids in arranging certain items on a canvas layout and thatprovides visual feedback to the user which gridlines and items or setsof items have priority for snapping to the grid, to one another, or toone another and the grid.

FIG. 48 is an elevation view of a double page canvas layout 4800supporting dynamic control over gridline snap-to functionality. Layout4800 is analogous to a project such as a photo book served to a clientfrom a server connected to a network, the server having access to leastone processor and a data repository, the server including anon-transitory physical medium including software running from thenon-transitory physical medium.

In addition to establishing a client-server connection between theserver and at least one user-operated computing appliance connected tothe network, and initiating and maintaining an active data sessionbetween one or more users involved in project creation and or in projectediting through a graphics user interface (GUI), the software, in thisembodiment, provides functionality to enable dynamic system control ofgrid line assignments for snapping to certain items.

Canvas layout 4800 includes two pages, each exhibiting a grid 4814 a anda grid 4814 b composed of horizontal and vertical gridlines. Grid 4814 aor b is analogous to grid 2103 of FIG. 21 described further above inthis specification. In this example, certain grid lines are prioritizedfor snapping to certain assets placed on the canvas. A pattern of blackarrows surrounding the perimeter of grids 4814 (a,b) indicateshorizontal and vertical gridlines that are prioritized as describedabove.

Grid 4814 a spans the first page of layout 4800, while grid 4814 b spansthe second page. In this example, grids 4814 a and 4814 b are alignedand prioritized relative to snap-to gridlines uniformly as a singlelayout. That is to say that the pattern of snap-to gridlines identifiedby the arrows is uniform and symmetrical across the layout. This isindicative of a two page spread where the same number of identical setsof assets is placed on both pages and in a mirror image arrangementrelative to both pages. It is noted herein that in other embodiments,grid 4814 a may be prioritized differently than grid 4814 b withoutdeparting from the spirit and scope of the present invention.

Grid 4814 a supports a photo slot 4801 and a photo slot 4804. Grid 4814b supports a photo slot 4802 and a photo slot 4803. In this example,photo slots 4801 and 4804 are snapped to the grid and in alignment withone another. The photo slots also occupy symmetrical positions on thegrid relative to the overall grid space. To position photo slot 4801, auser may drag photo slot 4801 towards the upper outside corner of grid4814 a. Photo slot 4801 has snap-to guides provided on at least twosides in this example. As photo slot 4801 is dragged nearer to thecorner, visual indication of the prioritized grid lines would becomehighlighted and a target line extending from the center of thehighlighted grid pattern and the center of the photo slot might appearindicating to the user that the photo slot will snap into that positionon the grid. Photo slot 4804 is prioritized to occupy the same space inthe lower outside corner of grid 4814 a. Thus, the photo slots aresymmetrically aligned and spaced apart from one another relative to theoverall grid space.

The same thing is true at the opposite end of the layout relative tophoto slots 4802 and 4803. The system aided by software may dynamicallyassign snap-to priority to the appropriate gridlines based in part ofthe size and shape of the photo slots. For example, two identical photoslots are logically laid out in a symmetrical pattern, occupying a spacethat is prioritized for those slots. Photo slot 4801 is aligned with andspaced apart from photo slot 4802 relative to prioritized grid lines ofboth grids where the gridlines are aligned at the grids intersection inthe center of layout 4800. The mirror image effect is produced in thisexample with the prioritized arrangement of four photo slots placed onthe canvas.

In one embodiment, the system knows the exact number of identical photoslots that will be placed on the two-page canvas. In this embodiment,certain theme-based layouts have a default number of identical photoslots. During an automated remix, however, that number and photo slotattributes may change. The system dynamically reassigns the priority tothe gridlines for a new special pattern of photo slots. For example,photo slots 4801 and 4804 might occupy space in the inside corners ofgrid 4814 a instead of the outside corners. In another example, theymight occupy opposite diagonal corners. Moreover, addition of otherassets may subsequently cause the system to reassign gridline priorityfor enabling another arrangement that includes prioritized assignmentfor snapping the new items to the same grid.

Grid 4814 a includes a text box 4805. Grid 4814 b supports an identicaltext box 4806. Text boxes 4805 and 4806 provide caption space fordescribing the photos in photo slot pair or set 4801 and 4804, and photoslot pair or set 4802 and 4803. In this example, a user is dragging textbox 4805 toward the outside edge of grid 4814 a. In one embodiment, twohorizontal grid lines are prioritized as targets for snap-to guides (notillustrated) on the top and bottom edges of the text box. In thisembodiment, at least one additional vertical grid line is prioritized tosnap to the inside edge of the text box to secure its position bothhorizontally (vertical grid line) and vertically (horizontal gridlines).Text box 4806 is snapped into position between photo slots 4802 and4803. Text box 4806 has vertical alignment and the same width as thephoto slots and is equally spaced between them.

In one embodiment, certain items are configured to snap to other items,which may or may not be gridline dependant. Photo slot 4801 and photoslot 4804 may have corners configured as snapping targets for snap toguides on the corners of text box 4805. The snap-to interface in thiscase includes a distance or space to maintain between the assets, forexample, the spacing of the text box between photo slot 4801 and photoslot 4804 to attain equal spacing. This distance or space is calculatedfor the position assignment based on the dimensions of the assets andthe overall grid space hosting the assets.

As the user moves the box in-between the photo slots, the corners of thephoto slots may light up or exhibit snapping “lines” as a visualindication of where the text box should rest in snapped position. Inthis case the snap lines include a distance specification along the gridso that text box 4805 snaps in a position that is centered between photoslots 4801 and 4804. If the user is building the layout from scratch,the grid assignments may be dynamically changed depending on item typeand item number. A change in number of identical items may force thesystem to adapt a different pattern for displaying the items. In thiscase there are only two identical text boxes and a user may understandintuitively where box 4805 should go by looking at the “snapped”position of box 4806.

In one embodiment, text box 4805 has a snap-to guide in the center ofthe box. As the user drags the box, when it comes close to photo slots4801 and 4804, a snap-to guide line appears indicating the snap-toposition for the center of the box. In another embodiment, a combinationof gridline targets and asset targets may be prioritized for snapping anasset with corresponding snap guides on the asset.

In one embodiment, gridlines of grids 4814 a and 4814 b are all ormostly snap-to targets when a user begins adding assets to layout 4800.As more assets of differing type are added, some gridlines may bede-prioritized, de-targeted, or deleted altogether to prevent snappingfor certain lines. In one embodiment, the system begins “turning off”certain gridlines for snapping based on breach of a population thresholdof assets added to layout 4800. This activity may coincide withselection of different arrangement patterns suggested by model forcertain assets. Arrangement or display patterns of assets on layout 4800may vary widely according to different numbers and overall footprints ofassets. For example, certain gridlines may be targeted for snap-to basedon the number of assets that are considered a set of assets. A set ofassets might be a number of identical assets or a number ofcomplimentary assets, or even a mix of identical and complimentaryassets.

In one embodiment, grid 4814 a supports a sticker 4808. Sticker 4808 isannular and therefore includes no points or flat edges. An asset with anannular profile may be somewhat more difficult to snap to grid 4814 a.In this case, sticker 4808 is snapped at the bottom and top peripheraledges at a specific angle of rotation. An identical sticker 4807occupies the inside corner space on the second page or grid 4814 b oflayout 4800. A sticker 4810 in the shape of a horse sits atop sticker4808 and is snapped in position on sticker 4808. In this case, stickers4808 and 4810 “snap together” at a specific location. Referring now tosticker 4807, snap-to guidelines become visible as a user drags thehorse sticker closer to sticker 4807. Stickers 4810 and 4811 do not snapto a gridline in this example, rather, to another sticker.

A sticker 4812 in the form of a soccer ball is configured with a snap-toguide to snap to a front hoof of horse sticker 4810. In this example, asa user drags sticker 4812 closer to horse sticker 4810, the perimeter ofsticker 4810 becomes highlighted or otherwise more visible to the userand a guideline appears between the ball and the snap-to position forthe ball on the horses hoof. An identical sticker 4813, in the form of asoccer ball, is illustrated in snapped position on horse sticker 4811.Grids 4814 a and 4814 b support a pattern of stickers 4809 (1-n), whichare in the form of stars.

Stars 4809 (1-n) are configured to snap to grid and to snap to oneanother to maintain an equal spacing in pattern arrangement. As a userdrags star 4809 (1) toward the pattern, a guideline appears snappingkeeping the item a specific distance from star sticker 4809 (2). Stars4809 (1-n) may be configured to snap to horizontal or vertical gridlinesas illustrated by the gridline arrows. Snap-to guides may be positionedon the bottom points of each star for snapping to horizontal gridlines.In another embodiment, a snap-to guide may be provided on the top pointof each star for snapping to vertical gridlines. In this embodiment, thesystem aids the user in generating a pattern of assets on layout 4800.In still another embodiment, a snap-to guide may be positioned at thecenter of an item. Different symmetrical and asymmetrical patterns forarranging assets on layout 4800 are, in one embodiment, modeled by thesystem.

In one embodiment, different patterns for asset placement using snap-toare modeled for theme-based templates using corresponding theme-basedassets. As a user places assets in the layout, an emerging patternbecomes apparent to the user by visual cue or indication to the userwhere the different assets snap onto the canvas and to one another insome embodiments. Specific pattern modeling may account for variousaspect ratios, asset numbers per page or layout, asset type, asset size,and asset combinations. As the user observes the interface, the patterntakes shape and the user learns the capabilities of the assets forsnapping to create an aesthetic pattern. In one embodiment, portions orpoints on a gridline may be activated or deactivated as snap targets forassets.

In one embodiment, there may be assets that are not configured forsnapping to grid. These asset types, including stickers with indefiniteboundaries such as swirl accent may be deemed unsuitable candidates forsnapping to the grid or to other items. In some embodiments, text boxesmay deemed unsuited for snapping leaving graphics assets like photoslots, stickers, and images snappable to the grid and or to one another.

In one embodiment there may be multiple gridlines on a grid that areactivated as targets for a number of different assets to snap to. Inthis case, the system may define a range of snap-to strength forgridlines and thereby give prominence to certain gridlines in attractingcertain assets. Intuitively, the system may learn from a user ormultiple users which types of items the users are more likely to preferfor snapping.

Photo Upload Optimizations

Referring now to FIG. 9, assets such as photos are uploaded into aproject by one or more users, and or obtained by the system from sitessuch as a Flickr™ or Facebook™ elsewhere on the network or any accountthe user is authorized to access. Bandwidth dependant devices such asmobile computing appliances connect to the service wirelessly usingwireless network services such as 3G network, Wireless Fidelity (WiFi),and so on. A user may upload photos to a project one at a time or inbatches so that a number of photos begin uploading but compete for theavailable bandwidth. If batch photos have a high resolution, they maytake longer to upload. Similarly, if a single photo is uploaded, theuser desires to see what the photo will look like in the interfacebefore finishing the upload, and this may be especially true for higherresolution images that may take longer to obtain through upload.

Referring now to FIG. 14, photo slot 1407 enables drag and drop photoaddition, photo panning, and various forms of editing capabilities. Inone embodiment, a user may interact with a control such as control 1410in order to initiate a process to perform a single upload into an emptyslot or to overwrite or overlay an existing photo in the photo slot. Inone embodiment, invoking interactive control 1410 via touch screen (somedevices) or mouse operation initiates an upload session associated withthe photo slot. Invoking button 1410 may initiate a “single photo”browsing interface that enables the user to select a photo for uploadfrom a local disk or peripheral device or to input a photo location oraddress of an asset held elsewhere on the network.

While the photo is uploading, the system aided by software detects thata user has invoked an upload control like button 1410 associated to asingle photo slot like photo slot 1407. Once the user has selected aphoto for upload, the system reads the file resolution of the photo. Ifthe resolution is higher than the minimum resolution for suitableprinting of the photo, the system may access or create a copy of theuser-selected photo and may compress the photo to a pre-designatedresolution suitable for acceptable quality in print. In this way, thesystem may first upload the compressed, lower resolution copy of thephoto and display that copy in a decompressed state in the photo slotwhile the higher resolution photo is still uploading.

The user may visualize the “system copy” of the photo in the photo slotahead of receipt and display of the original photo. The user maydetermine after seeing what the photo will look like at the very leastresolution deemed suitable for print quality that another photo shouldgo there instead. The user may cancel the upload and browse to anotherphoto. Using this method, the user might see system copy photos in morethan one photo slot on a canvas layout while two or more of the originalphotos are still uploading. In this case, the user may quickly selecteach of multiple photo slots on a canvas layout subsequently andinitiate uploads of a single photo to each slot resulting in amulti-photo or batch upload where each uploaded photo is addressed tothe appropriate targeted photo slot.

The above-described process is different than uploading a batch ofphotos first and then dragging and dropping photos into photo slots, forexample. This flexibility is important for users operating from mobileappliances that have high resolution photos for use in a project. Inthis embodiment, the system aided by software, replaces each “systemmade copy” of each photo by the original file after it is completelyuploaded and decompressed for display.

FIG. 49 is a process flow chart 4900 depicting steps for simulatingdisplay of an uploading photo according to an aspect of the presentinvention. Process 4900 assumes that a client is connected to theservice and has executed a project to work on. At step 4901, the systemdisplays the project executed by the client operating through a GUI. Atstep 4902, the user engages in editing or building on to the project.The project may be a photo book or some other project description withphotos included in the project.

At step 4903, the user determines whether or not to add a photo to aphoto slot. If at step 4903, the user decides not to add a photo, theprocess resolves back to step 4902 where the user continues in session.If the user decides to add a new photo into a photo slot, the userinvokes a photo upload or insertion control at step 4904. Invoking acontrol associated with the photo slot similar to control button 1410 ofFIG. 14 described further above, causes a browser interface to appearwhere the user may select a photo for upload or designate a photo foruse that is hosted elsewhere on the network at step 4905.

At step 4906, the system accesses the selected photo file whether it isstored locally (client device) or hosted elsewhere on the network. Ifthe file is hosted on a user account elsewhere on the network, thesystem accesses that file on behalf of the user and with userpermissions. If the file is stored locally, the user allows the systemaccess to the file on a local device. In one embodiment, a softwarecomponent is previously downloaded to the user's device to enablemanipulation of the file by the system while the user is connected tothe server.

At step 4907, the system determines if the accessed photo has a pixelresolution that is equal to or greater than a minimum resolutionrequired of the photo for suitable print quality. If at step 4908 thesystem determines that the photo resolution is not equal to or greaterthan the minimum value required for quality printing, an error messagemay be sent to the user at step 4908 that the selected photo forinsertion to the project does not meet the minimum quality standards fordisplay and print. If the system determines that the photo file is equalto or greater in resolution than the minimum value suggested for qualityprint at step 4907, the process moves ahead to steps 4909 and 4910,which may be performed concurrently. At step 4909, the system copies theasset and reduces the copy to the lowest possible resolution. At step4910, the system copies the asset and preserves the photo resolution ofthe asset.

At step 4911, the system initiates upload or transfer of the assetcopies created at steps 4909 and 4910. Both asset copies may becompressed for file transfer. One of the copies is lower in file sizebecause of a lower photo resolution. At step 4912, the system receivesthe lower resolution copy while the higher resolution copy is still inprogress of uploading or transferring to the system. At step 4913, thesystem decompresses and displays the lower resolution copy in thetargeted photo slot. The user may then visualize what the photo willlook like at least at the minimum pixel resolution suggested for printquality.

At step 4914, the system determines if the higher resolution copy of theasset has been received. If the system still has not received the higherresolution photo file at step 4914, the process is delayed at step 4915.During this delay, the user is free to move on to the next item in theproject for editing or building. If the system has received the higherresolution copy at step 4914, the system decompresses the photo file andreplaces the lower resolution copy in the photo slot with the higherresolution copy at step 4916. The lower resolution copy may be laterdeleted or purged from the system. The process resolves back to step4902 where the user continues to edit and or build the project. In thecase of client sourced photos, by compressing all of the images orphotos on the client device to a resolution sufficient for display atthe lowest acceptable print quality, the system is aided in getting allof these images uploaded before the client closes out of the applicationor otherwise terminates the session. If certain higher resolution copiesultimately fail to upload or are not received for any reason, the systemhas copies that are of acceptable quality for display and print.

In one embodiment, the process is also applicable for users on dedicatedbandwidth connections with time-constrained applications. If a user hasnumerous high resolution photos for upload, for example 300 or more, itmay take a long time even over the best bandwidth connections.Therefore, the system may compress those photos on the user device asdescribed further above with bandwidth constrained devices, and obtainlower resolution copies of these from the user device so that the usermay begin building the project immediately while the higher resolutionphotos are uploaded in the background. In one embodiment, the system mayprovide the user with a desktop application or component that enablesthe user to maintain a connection with the server for uploading thehigher resolution photos even if they log out of or close their browserapplication.

Print Resolution Visualization on Digital Display

Referring back to FIG. 9, assets such as photos are uploaded into aproject by one or more users, and or obtained by the system from sitessuch as a Flickr™ or Facebook™ elsewhere on the network or any accountthe user is authorized to access. Bandwidth dependant devices such asmobile computing appliances connect to the service wirelessly usingwireless network services such as 3G network, Wireless Fidelity (WiFi),and so on. A user may upload photos to a project one at a time or inbatches so that a number of photos begin uploading but compete for theavailable bandwidth. If batch photos have a high resolution, they maytake longer to upload. Similarly, if a single photo is uploaded, theuser desires to see what the photo will look like in the interfacebefore finishing the upload, and this may be especially true for higherresolution images that may take longer to obtain through upload.Moreover, digital prints reveal a print resolution that computingdisplay screen cannot deliver. It is desirable then, that users are ableto visualize to some extent the higher resolution factors associatedwith printed materials on the display screens associated with theircomputing appliances.

FIG. 50 is a process flow chart 5000 depicting steps for simulatinghigher photo resolution on a display screen for user visualization. Itis assumed in this process that the user is connected to the service andis in session with the server through the GUI. At step 5001, the userexecutes a project such as a photo book, for example. At step 5002, thesystem renders the layout for display. Before display, the systemreduces the size of the layout by some pre-determined or dynamicallydetermined factor at step 5003.

The purpose of the size reduction at step 5003 is to simulate a higherpixel resolution than would be visualized at 100 percent or greater sizefactors. It is noted herein that a canvas layout incorporates one or twopage spreads and there may be multiple layouts in a photo book. However,the system may upon reducing the first layout viewed, apply thereduction to the other layouts in the project. This may occurdynamically as a user turns pages in the photo book or in apre-determined fashion in the background independent of whether or notthe user will view certain pages in the project. A factor ofmultiplication may be used in reducing layout size such as by 0.25 or by0.5, 0.75 etc.

At step 5004, the system creates a texture mask of the size-reducedlayout for display. The texture mask functions to “ratchet up” thevisual resolution simulation by adding dots and other idiosyncrasiespresent in the print output data. At step 5005, the system displays thelayout at the reduced size with the texture mask.

Tracking Theme-Based Digital Assets

FIG. 51 is an architectural diagram of a network 5100 supportingtracking of theme-based assets according to an embodiment of the presentinvention. Network 5100 is the Internet network in a preferredembodiment. Network 5100 includes any connected sub-networks includingcarrier networks and access networks both wired and wireless.

Users such as one operating computing appliance 5106 create custom photoproducts for themselves and for others. It is desirable that users suchas one operating appliance 5106 can easily find templates and relatedproducts. Network 5100 includes a Web server 5101 and a runtime server5104. Web server (WS) 5101 hosts a Web site 5102 where users may loginto to begin a project or to edit, build, and purchase one or moreexisting products. Selection of an existing project causes a serverredirect to a runtime server (RS) 5104. Runtime server 5104 includesaccess to least one processor and a data repository, the serverincluding a non-transitory physical medium. SW 5105 running from thenon-transitory physical medium provides several functions in addition tofacilitating project creation, editing and transacting.

RS 5104 has connection to a data repository 5113 containing stylisticthemes, which are not tied to or related directly to any particular usecase or category of product, but are tied specifically to particulardesign esthetics. For example, a theme could be “Modern Swirls”. RS 5104has connection to a data repository 5114 containing templates. Eachtemplate in repository 5114 is tagged with multiple (list) tag words andis associated with a theme according to a data model described later inthis specification. RS 5104 has connection to a data repository 5115containing tags arranged in tag groups. In this embodiment, eachtemplate in template repository 5114 is associated with multiple tags,some of which may be incorporated into tag groups in repository 5114. Anexample of a tag group might be “Girl, Photo, Baby or BirthAnnouncement”.

SW 5105 provides a search engine and search engine interface for usersthat enable a more streamlined search for digital assets, in this case,templates. Appliance 5106 has a GUI 5107 displayed on screen thatincludes a search engine interface in the form of a search field visiblein this example. Templates in repository 5114 have an associatedstylistic theme. Moreover, each template is tagged with multiple tagwords relative to template content and template functions. These tagsmay be further formed into tag groups in repository 5115. GUI 5107 showsa search input field a user may leverage to search for templates basedon tag words input into the engine. Inputting tag words may also returnmore tag words formed into a group of tag words that are associated withtheme-based templates. Users may search for templates in template groupsby using the tag words as input, for example by entering a tag wordstring such as “photo, girl, birth, announcements”.

Network 5100 supports a computing appliance 5111 operated by anadministrator (ADMIN) or knowledge worker. Appliance 5111 has a GUI 5112and a search interface presenting a search input field. An administratoror knowledge worker may use the search interface to find templates foradministrative purposes such as changing, deleting, or adding tag wordsto particular templates, and so on. A user searching for templates usinga tag group may be served multiple different templates under a sametheme, the templates matching the tag words used in the search.

In one embodiment, the data model enables cross-merchandising products(up selling) within a theme based on commonalities between the products.For example, when a user initiates a transaction, the system may offerto upsell a user purchasing a product with the tag words “pink, baby,shower, invitations” with the product “pink, baby, birth, announcements”from the same theme. Network 5100 supports a server 5108 that includesat least one processor and a data repository, the server including anon-transitory physical medium hosting software in the form of an upsellengine 5109. Server 5108 includes a data repository 5110 containingupsell rules adapted to prompt the system to make automated offers forother or additional products based on commonalities between the productsevidenced by the tag word groups related to template groups of a sametheme.

FIG. 52 is an entity relationship model 5200 depicting relationshipsbetween tags, tag groups, and templates that are theme associatedaccording to an embodiment of the present invention. Templates 5201include all templates (1-n) known to the system. The exact number oftemplates in the system may rise or fall based on template creation,template retiring, etc. In this example, an administrator like oneoperating computing appliance 5111 of FIG. 51 may create multiple tagwords for a particular template, the tag words describing attributes ofthe template including functions of the template. Therefore, in the datamodel, one template 5201 may have many tags 5203. Tags 5203 (1-n)represent all current tags associated with templates known to thesystem.

Templates are not grouped or isolated by theme. Themes 5205 include allthemes (1-n) that are known to the system. Therefore, a theme 5205 maybe associated with many templates 5201. Tag groups 5204 (1-n) representgroupings of tags formed from tags 5203 (1-n). A tag group 5204 has manytags 5203.

In a preferred embodiment, an administrator or knowledge worker createsa list of tags for each individual template. The tags aid in thedescription of different aspects of the template. For example, atemplate could be related to the tags “#CCEEFF (color), ‘baby’(subject), ‘announcements’ (usecase)”. It is noted herein that each tagreference includes the type of the tag indicated in parenthesis.Knowledge of tag type enables system software and personnel to performdata analysis across the different tagging domains. For example, thesystem aided by software or a knowledge worker operating from aconnected computing appliance may report current revenues accrued over aperiod of time by the tag subjects baby, women, man, boy, and girl.

The tags may be grouped to form tag groups 5204 like a tag groupcontaining the tags girl, photo, birth, and announcements for example.The tag groups are joined with the templates through the themes.Therefore the system may cross sell merchandise (template based) withina theme based on the tags in a group of tags. To illustrate a use case,consider that the system determines to present different color optionsto a user for coloring a project. The system could present usertemplates that match along all tagging domains except for the color tagtype.

With respect to upsell engine 5109 described further above, upsell rulesmay be created and based on tags 5203 (1-n) and stored for laterconsultation like rules 5110 in repository 5110 of FIG. 1. For example,if a client is purchasing a “wedding invitation card”, the system mayoffer an upsell based on an upsell rule that points to other productsavailable in the same theme like a “wedding guest book” by performing alookup or search against a subset of tags common to both producttemplates. Moreover, there may be multiple different templates to selectfrom to offer an upsell, each representing a varied product adhering tothe same theme and tag grouping.

FIG. 53 is a process flow chart 5300 depicting steps for offering anupsell to a client involved in a transaction. At step 5301, a userrequests a transaction for a product. The system, aided by softwarerunning from a non-transitory medium on the server, identifies (IDs) theproject and associated theme. The system retrieves the tags associatedwith the project template at step 5303. The system searches for moretemplates of the same theme using the retrieved tag words (Tag Group) assearch criterion at step 5304.

At step 5305 the system determines if there are any related templatesunder the same theme that match the input tag word string. If the searchreturns no related templates at step 5305, the process skips to step5311 to finish the original transaction. If the system returns relatedtemplates at step 5305, the system may consult upsell rules at step 5306to determine if an upsell recommendation should be made to the user. Atstep 5307, the system determines if there is an upsell offerrecommendation. If at step 5307, there is no recommendation for upsellduring the pending transaction, the process moves to step 5311 and theoriginal transaction is finished.

If the system receives an upsell offer at step 5307, the system displaysthe upsell offer to the user or otherwise notifies the user of theupsell opportunity at step 5308. At step 5309, the system determineswhether or not the offer is accepted by the user. If the upsell offer isnot accepted by the user at step 5309, the process moves to step 5311 tofinish the original transaction. If the upsell offer is accepted by theuser, the system may modify and obtain confirmation of the modifiedtransaction at step 5310. For example, the user purchasing a weddinginvitation card may also be asked to purchase a wedding photo book. Theupsell product may be discounted for early upfront purchase, or paymentfor the book may be delayed until the book is completed by the user andis ready for purchase and print.

Dynamic Bundling of Editing Tools

Referring now to FIG. 10 through FIG. 15, editing tools are providedwithin the GUI editor, also referred to as the Mixbook editor for userswho are building and editing projects. Among the tools provided arenovel color choosing applications, sizing applications, photo editingtools, among other useful tools that user may invoke through the editor.Notwithstanding the different levels of experience users have in editingthrough the GUI, users may also have preferences in certain editingtools and methods of editing and collectively, are working on verydifferent types of projects spanning image-based and or text-basedproducts they have initiated.

FIG. 54 is a process flow chart 5400 depicting steps for dynamicbundling and delivery of editing tools in an editor based on userpreference and project analysis. Process 5400 assumes the user is loggedonto the server and is in session. At step 5401, the user executes aproject to work on. At step 5402, the system identifies (IDs) theproject and accesses the project and project content.

At step 5403, the system analyzes the project including content stateand editing history. In this process, the system analyzes the templateand template functions, project content population state, and editinghistory. At step 5404, the system assesses current editing needs basedon the analyzed data. Certain editing tools, features and functions mayor may not be needed based on the type of project template underlyingthe project; the state of the existing content within the project; anddiscovery of historical norms of editing within the project spanning oneor more previous editing sessions engaged in by the user relative to theproject.

For example, if the project template underlies a photo book, certaintools may be indicated by default for photo book editing, whereas if thetemplate is an invitation card, certain other editing tools and featuresmay be indicated in a default setting. If the project is full of contentthat has been repeatedly edited, the system may access the list of thelast editing tools and features that were bundled together and presentedto the user.

In one embodiment, a user may have stated preferences for certain typesof editing tools and features for specific types of projects. In thiscase, the system may also access these user preference settings, whichmay be specific to template type. In another embodiment, the system mayanalyze the recent history of user editing tools and feature use andanalyze statistics of actual editing activities on the project and forsimilar projects.

At step 5405, the system retrieves editing tools and features, includingplug-in components and extensions that are indicated as needed by thesystem as a result of analysis performed in steps 5403 and 5404. At step5406, the system loads the retrieved editing tools and features,including any indicated plug-ins and extensions into the user'stoolbar(s) and GUI interface. Plug-ins may include external toolsadapted to work in the editing interface as a plug-in component.Extensions may include extensions to a posting board or textcommunication interface, extensions to other graphics editing programs,or extensions to social interaction pages. At step 5407, the systemdisplays the user's project in the GUI with the customized tool bundle.

In one embodiment, the user receiving the customized bundle may furthercustomize the editor based on planned future activity or current needsusing an update feature or settings feature for adding content in theform of different editing tools and features. The variety of editingtools and features delivered to the user might vary considerable basedon system findings and needs assessment. For example, if the userselects a blank template and has not yet added content, then the systemmay provide a fuller suite of editing tools and features in anticipationof their use on added content. On the other hand, if the user selects aproject that is fully populated with content that has already beenedited and the user is currently working on text for the last fewsessions, perhaps photo editing tools and features would be omitted fromthe custom bundle.

In one embodiment, longer term editing history could be analyzed andstatistics relative to tool use and feature use percentages might becompiled for use in steps 5403 and 5404. For example, if usage of photoeffects filters is constantly below a certain threshold, such tools maynot be bundled in by default but made available if requested by theuser. In some cases a user's preferences may be for the system to setcertain photo effects, text effects, and content management tasks. Insuch a case, those particular tools and features controlling sucheffects and enabling such tasks may be omitted from the custominterface, but made available through a settings configuration interfaceadapted for the purpose of adding, deleting, or toggling certain editorsettings or features.

It is noted herein that a user may have more than one project inprogress on the system. Therefore, the user may receive a differentbundled editor depending in part of the project type. Likewise, if auser receives a bundled editor at 5407 and during subsequent work thesystem observes a need for a tool not originally delivered with thebundle, the system may suggest or recommend additional editing tools orfeatures to the user where if accepted, the tools or features aredelivered as plug-ins to the interface. There are many possibilities.

Client Cache Management of Data

FIG. 55 is a process flow chart 5500 depicting steps for caching datalocally on the client for persistence according to an embodiment of thepresent invention. FIG. 56 is a process flow chart 5600 depicting stepsfor persisting data at a subsequent user session with the same project.

Referring now to FIG. 55, it is desirable that users do not lose anydata in the event of session termination while working on project. It isfurther desirable that session performance over the server/clientconnection established is optimal in terms of retrieving and presentingor rendering the latest assets or content in a user's project.Therefore, the inventors provide a data persistence method foroptimizing content retention and content rendering that involves storingall changes a user makes that would result in a server call in localcache and persisting those changes to the server at certain intervals oftime or in the event of un-planned or planned session termination.

Chart 5500 represents a user's first data session where components areestablished for local caching and persistence of user changes resultingfrom user activity in the session. At step 5501, a user initiates a newproject or executes an existing project to build onto and edit. At step5502, the system identifies and gets the project template and theassociated project content. Content includes text assets and graphicsassets. At step 5503, the system displays the project for the user.

At step 5504, the system may download and install a cache memorycontroller to aid in sequestering some cache memory for local datastorage. In one embodiment, users are prompted about whether or not auser wants to store data locally for persistence in session. In anotherembodiment, local cache is leveraged on the client appliances bydefault. At step 5505, the cache controller component sets up andmanages a portion of local cache memory. In one embodiment the memoryallocated for a “data reservoir” is random access memory (RAM).

At step 5506, the system may cache any changes in content made by theuser. For example, a user may ad a photo, a caption, may move an item,swap one photo for another, etc. In a preferred embodiment, every useraction that results in a server call is cached locally in a manner thatpreserves the latest “version” of the data class and state in cache witholder changes overwritten in cache where appropriate. This data may bepersisted to the server at X intervals.

At step 5507, the system may determine if the current session will end.A session may end in a planned fashion or a session may endunexpectedly. This determination may be enabled by such as an “end ofsession” or a “logout” routine informing the system that the sessionwill end or is ending by action of the user. If at step 5507, the systemhas determined that a session will end as initiated by a user, thesystem may persist (upload) the locally cached changes to the server atstep 5509. If the session has ended unexpectedly at step 5507, then step5509 may occur when the user re-establishes the session. It is notedherein that the user actions cached locally are rendered locally on theuser display so that the user sees all of the changes locally even inthe absence of persistence to the server.

If the system determines the session will not end at step 5507, theprocess may move to step 5508 where the system determines if a datapersistence interval has arrived. A data persistence interval may be anyinterval of time the system uses to persist the data in cache to theserver. If the system determines in step 5508 that a persistenceinterval of time has arrived during session, the server may persist thelocally cached changes to the server. If the system determines that apersistence interval has not yet arrived at step 5508, the processresolves back to step 5506 where the system continues to cache the useractions locally.

In one embodiment, the system may, in addition to caching user actionsresulting in server calls, cache certain project components locally onthe user's machine. In this embodiment, the system may use a taggingschema to version content cached locally and stored at the server. Forexample, if a user's project has items that the user does not generallymodify like a backdrop, stickers, or other content, that content may becached locally and tagged for version. The user may then have a tag listcached locally that itemizes the content items and last modifiedversions.

Referring now to FIG. 56, the user executes a project at step 5601. Atstep 5602, the system gets the project identification (ID) and retrievesa project-associated tag list from user cache. The tag list representsthe items cached locally and the current version and state of thoseitems. At step 5603, the system locates the project template andassociated content. In one embodiment, some assets are tagged forcaching while others are not tagged for caching. System rules for whatclasses or types of content and how much of that content should becached at the user appliance in-between data sessions depends in part onthe project content type, available bandwidth, and the capacity of thelocal cache memory reserved on the user appliance. The system may relyon special asset rules for tagging and local caching.

At step 5604, the system determines against the tag list if an iteminherent to the project is tagged (cached locally) or not tagged (storedat system level). If at step 5604, the system determines there is atleast one item not tagged for local cache, the system gets the item orun-tagged items from server storage. However, if the system determinesthat one or more items are tagged, then those items are rendered locallyfrom the user cache memory. The system may continue to tag and cacheitems to the user's cache depending on rules, device capacity,bandwidth, etc. For example, if background stickers for a project aretagged assets for local cache, introduction of a new sticker may resultin tagging the new sticker and caching the new sticker locally with theother tagged stickers. If the new content is a text update, a styleupdate, or size update, etc. made to text associated with a cached textasset, the system may overwrite the input in cache associated with thatasset. In one embodiment, the text input may be cached while theassociated asset is stored sever-side.

In one embodiment, the tag list informs the system which project contentis cached and which project content needs to be served by the system.The tag list can be stored locally and retrieved by the system or it maybe stored at system level without departing from the spirit and scope ofthe present invention. The tagging method may be practiced inconjunction with caching user actions in editing as was describedfurther above.

At step 5604 the system determines as it accesses project content, ifthat project content is tagged or is not tagged. If at step 5604, thesystem determines that the project content is not-tagged, the systemgets the server-based project content. If the system determines thatproject content is tagged, the system gets the content from the cachememory on the user's appliance. At step 5607, the system displays thecontent. In a preferred embodiment for most project types, some contentis system served (not tagged) and some content is displayed from cache(tagged) and not served by the system.

It is noted herein that in one embodiment, local caching of content ismade available to users who are working on projects in anon-collaborative way wherein only the user accesses the project. Inanother embodiment, local caching of content may be made available tomultiple collaborators working on a project. To illustrate further, ifthere are three users working on a project A and all three have a cachedversion of asset C, the system might maintain the latest “version” X ofasset C as modified or created as a result of an action by any one ofthe multiple collaborators. Upon subsequent session initiation with thesame project, the collaborator that made the last modification resultingin tagged asset C version X, for example, has the asset version uploadedfrom cache. The previous version of asset C, which may be a tagged assetcached on the appliances used by other collaborators, may be deletedfrom cache as obsolete while the system serves the latest version from asystem copy of asset C version X.

When the collaborators end sessions working on the project, each mayhave the latest version of asset C downloaded to cache if the asset istagged for cache. Versioning enables the system to determine if thecached asset was replaced, deleted, or modified in collaborativeprojects. It is also noted herein that in collaboration, users may beworking on different parts of a project at different periods of time.Each user that logs into the project is presented with the latestversions of the assets, from cache and or from server-accessible datasources associated with the portion of the project they are viewing indisplay such as the visible assets in a two-page spread of a multi-pagephoto book.

Content Proposals Based on Gathered Information

In one embodiment of the present invention, the system may propose orrecommend content to users working on projects based on current projectand project content information gathered during project building andediting sessions.

FIG. 57 is an architectural overview of a network 5700 supportingcontent proposals based on information gathering according to anembodiment of the present invention. Network 5700 is the Internetnetwork in one embodiment and is further illustrated by a networkbackbone 5701. Backbone 5701 represents all of the lines, equipment, andaccess points that make up the Internet network as a whole, includingconnected sub-networks. Therefore, there are no geographic limitationsto the practice of the present invention.

Backbone 5701 supports a user depicted as a computing appliance 5702.Appliance 5702 may be a mobile computing appliance connecting through awireless network or a wired appliance connecting through a wirednetwork. It is assumed in this example, that the user operatingappliance 5702 has connection to runtime server (RS) 5707. Runtimeserver 5707 includes at least one processor, a connected data repository(not illustrated), and a non-transitory physical medium hosting an assetrecommendation engine (ARE) 5712 analogous to ARE 2905 of FIG. 29.

ARE 5712 is already adapted to make content recommendations to usersaccording to analyzed usage counts, friend relationships among thecontent creators, and content characteristics, like width and height, orcontent meta-data such as photo data. ARE 5712 may be integrated with SW5711 running from the non-transitory physical medium of the server. SW5711 is adapted to gather information in a project being worked on andthen to analyze the information gathered to develop a primary list ofkeywords used in the project or relevant to the project. SW 5711 isfurther adapted to perform searches of one or more data sources forimages and other assets using one or more of the primary keywords and topresent one or more digital assets found or returned in the search tothe one or more users working on the project. SW 5711 may includeseveral components for gathering and mining information including textreader or text parsing components, snapshot components, screen scrapingcomponents, pattern recognition components, file reading components,etc.

Network backbone 5701 supports a server 5708 adapted to returntheme-based assets stored in a connected repository 5709. Theme-basedassets include stickers, shapes, frames, decorative borders, and otherimagery available within the system that is artistically and digitallyassociated with a project theme known to the system. Server 5708 hosts asearch interface 5710, which is machine operable via another server suchas RS 5707 executing SW 5711. In one embodiment, SW 5711 may initiate asearch of theme-based assets stored in repository 5709 by submitting oneor more keywords from the primary list of keywords developed frominformation gathered from a user's project. Assets returned by server5708 are rendered on the display of the user's appliance 5702 astheme-based assets 5722. Theme-based assets 5722 may appear asthumbnails, icons, or some other visual indication in a side bar 5704 inthe GUI 5703.

In one embodiment, SW 5711 may through extension or API, access thirdparty asset libraries to access photo and text assets for use inclients' projects. In some cases, the assets returned and kept or usedby the system may be purchased from the third-party systems. In thisembodiment, network backbone 5701 supports a server 5714 adapted toserve photos stored in a connected image repository 5716. Photo library5716 may be a public or private library. Server 5714 hosts searchinterface 5718, which may be analogous in structure and function tosearch interface 5710. Server 5714 and assets stored in repository 5716may be owned by a third-party as previously described.

Network backbone 5701 supports a server 5713 adapted to serve textassets stored in a connected repository 5715. Text library 5715 may be apublic or private library. Server 5713 hosts a search interface 5719,which may be analogous in function and structure to search interfaces5718 on server 5714 and interface 5710 on server 5708. In oneembodiment, text library 5715 contains famous quotes or sayingsincluding metadata about the quotes or sayings such as authors, timecoined meaning, etc. In one embodiment, photo library 5716 includesphoto metadata including, place (geo-location), time photo was taken,subject identification, and other description. In this embodiment,picture file data may also be available such as pixel resolution,height, width, file extension, etc.

SW 5711 may initiate searches for photos in photo library 5716 throughsearch interface 5718 by submitting one or more keywords from a list ofprimary keywords formed from information gathered in a client's project.Photo assets returned to server 5707 may be displayed in sidebar 5704 asphotos 5720. SW 5711 may likewise initiate a search of text-based assetsin text library 5715 through search interface 5719 on server 5713. Textassets returned to RS 5707 may be displayed in sidebar 5704 of interface5703 as text assets 5722. Indication of the returned text assets may beindicated by thumbnails, short title blocks, expandable summary blocks,or other visual indicators.

In this embodiment, the user operating appliance 5702 is working on aproject displayed in interface 5703 as a two page spread containingproject content in the form of photos like photo 5705 and associatedcaption boxes, and text columns, like text column 5706. Photos thatmight be used in a project include may include metadata like geographiclocation information, time and date of the photo, identification of thephoto subject, etc. Text entries like captions, titles, and textdescriptions or descriptive writing in a project may be mined forinformation that may reveal content including primary theme, repeatedsubject descriptions, opinions, stating of ideals or principles, and soon.

As a user works on a project and adds content, the system may mine theproject content for information. This process can occur when users arebuilding and editing a project online or it may be accomplished offlineby mining the data in the project database, or both. SW 5711 mines datafrom project 5703 and looks for prominent subjects, ideas, themes,places that are prevalent data in the project and that might be repeatedthroughout the project. SW 5711 then generates a list, pool, or cloud ofspecific keywords or phrases that characterize the prevalent content inthe project and that may be later used as search criteria by the systemto return specific assets that might be relevant to various aspects ofthe project.

Tags and meta-tags may be included in the project data mined as well asinformation that is recognized such as facial recognition of persons inproject photos, technical data such as styles, fonts, photo dimensions,and the like. In one embodiment, information gleaned from a project maybe broadened or generalized and expanded through associating data withother data or external data providing more information that might bediscovered or inferred by the system from the initial data analysis. Forexample, if there are several photos of places in a project about a tripto Europe, the system may analyze the photo metadata and text associatedwith those photos and generate prominent or primary keywords that can beused singly or in combination to retrieve more photos showing Europeanthemes or more places, famous quotes from European figures in history,or theme-based stickers or shapes that might be related to Europe.

If a project contains text that is about a subject like the museum ofnatural history in New York City, and the text is prevalent throughoutthe project, keywords might be generated from the text that can be usedto search or assets the may relate in some measure to the theme museumof natural history in New York. The assets may be presented to the useras theme-based assets 5722, text assets 5721, and photo assets 5720. Theuser may browse and select such assets for inclusion into the project.In one embodiment, the system extends the class of mined data to audiodata that might be linked to or embedded within the project.

In an offline mode, the system may routinely access project content inopen projects and mine the existing data perform one or more datasearches using the appropriate keywords, and have new content forpresentation to the user at the next session. There are manypossibilities such as accessing data from external sources and proposingaudio data. In one embodiment, the system may search for backgroundmusic to play along with an automated slideshow of the project where thebackground music selections are compiled based on information retrievedfrom the project. For example, in a photo book about a vacation inHawaii, information from the project such as place names, regions,island identifications, etc. may lead to keywords that can be used tosearch a database of Hawaiian music that returns music clips orselections that relate to the subject and theme of the project and tomore granular aspects of the project in some cases. As a projectreceives more and more content, the system may use the added informationto search for yet additional content that might be presented to users.

Selection and Management of Image Assets

In one embodiment of the invention, a photo picker application isprovided that includes a photo container or tray component that displaysselected photos in an editable state, in their correct aspect ratios(portrait, landscape, square), and reveals the current order of theselected photos in the project.

FIG. 58 is an exemplary screen shot of a graphics user interface 5800supporting photo selection and management for project enhancementaccording to an embodiment of the present invention. GUI 5800 has animage-based project 5801 opened and displayed within the interface. Itmay be assumed in this example, that project 5801 is a multi-page photobook opened to the first two pages. Project 5801 includes a number ofphoto slots arranged on the first two pages of the project. These are aphoto slots 5805 and 5806 on the first page and photo slots 5802, 5803,and 5804 on the next page in this view. Photo slots 5802 through 5806are adapted to contain photos for display.

In current tradition photo pickers enable a user to select photos foraddition to a photo project, however the user has to engage inadditional steps to preview the selected photos, and to add those photosinto a project. In one embodiment of the invention, a photo pickerapplication is provided that includes a photo container or tray 5808.Photo tray 5808 is adapted to contain and display photos that have beenselected by a user from a list or pool of available photos displayed ina photo browser interface 5807. Browser interface 5807 may displayphotos that are held locally on the user's computing appliance, on aperipheral memory card, flash stick, or other portable medium, or photoslocated elsewhere on the network.

Window 5807 displays 25 photos in this example and is scrollable toreveal additional photos. In this case, each displayed photo includes asmall selection box that may be checked by a user to indicate photoselection or unchecked to indicate non-selection. In this embodiment,there are eight photos selected by the user for addition to the project.The eight selected photos are loaded into photo tray 5808 as selectedphotos 5809 (1-n). Photo tray 5808 is scrollable to reveal more selectedphotos beyond what may be viewed in the tray window space. Browserinterface 5807 and tray 5808 may be expanded in size to reveal moreworkspace and thus more photos.

In one embodiment, selecting photos for addition to an image-basedproject leads to a default order of photos that will be added into theavailable photo slots in the default order page by page in the projectuntil all of the photo slots contain a photo. In this example, thephotos 5809 are numbered 1 through n. In this example, the top-leftselected photo in interface 5807 correlates to photo 5809 (1) in phototray 5808, the selected center photo in the next column of photoscorrelates to photo 5809 (2) and so on. Other default sequence ordersfor photos added to project 5801 may be observed without departing fromthe spirit and scope of the present invention.

In one embodiment, the SW may include logic for suggesting an order ofpresentation of the selected photos in the project based on certainattributes or a combination of several attributes of the selectedphotos. All of the photos are displayed as thumbnails in browserinterface 5807, but in tray 5808, they are displayed in their correctaspect ratios. In one embodiment, aspect ratios might be considered inphoto order as suggested by the system during a remix. In oneembodiment, browser interface 5807 (photo selector) and tray 5808 (photocontainer) are adapted for use on mobile computing appliances such as asmart phone, android device, or similar appliance that may have somewhatlimited display screen dimensions. However, the same configuration mayalso be used in the case of full size displays typical with largerlaptop and desktop appliances.

In practice, when a user selects a photo from within interface 5807,that photo is copied and transferred into photo tray 5808 as an editablethumbnail. A user may manually change the order of photos in tray 5808by selecting them and dragging them out of line and dropping them intoanother position in the line, or by cutting them and pasting them atanother position in the line of photos. In this example, no photos havebeen added yet to project 5801. However, each photo slot may be adaptedto reveal the default order of the photos relative to the photo slots oneach page. In this example the five photo slots will contain the first 5selected photos 5809 (1-5) by default order. If the user adds anotherphoto slot to one of the pages, the SW detects the addition andreadjusts the order of selected photos. For example, if the user adds anew photo slot to the first page beneath photo slot 5805, then it wouldbe the target receptacle for photo number 5809 (2) and photo slot 5806would then be assigned to display photo 5809 (3), etc.

In one embodiment, a user may mouse over or right click any of thephotos in tray 5808 in order to view properties like aspect ratio, photodimensions, pixel resolution, main color theme, color saturation, andother similar properties or attributes of the photo. In one embodiment,a user may right click any of photos 5809 (1-n) in order to bring up thephoto in a photo editing application. In one embodiment, the user mayview a larger image of any of the photos 5809 (1-n) by right clicking onthe photo and selection that option from the right-click menu. Thecombination of components (browser list and tray) enables users tochoose photos as well as preview photos and always see the photos theyhave selected even if they scroll past them in the browser listinterface.

Mosaic Design Patterns for Project Covers

FIG. 59 is a process flow chart 5900 depicting steps for generating amosaic pattern of photos for an image-based project according to anembodiment of the present invention. Process 5900 assumes that a userhas built and edited an image-based project online and is ready to orderthe finished product. At step 5901, the user purchases the project. Thisstep is not specifically required to practice the present invention, butbecause this process involves die-cutting, likely the die-cuttingprocess and printing process would be performed by a same vendor afterproduct purchase. In both instances (die-cutting and printing), thesystem prepares the project before routing a digital order for finalprocessing (die-cutting and printing).

At step 5902, the system retrieves the project from the project databaseor other suitable storage. An image-based project includes a number ofimages, each typically presented in a photo slot, photo well, or in somesimilar image container. At step 5903, the system analyzes the projectphotos. It is noted herein that each photo presented in a photo slot orphoto container may not be cropped to fit the container boundaries. Thatis to say that the portion of the photo extending outside the boundariesof the container is not visible to the user. However, in one embodiment,a photo may be cropped to fit within the container boundaries withoutdeparting from the spirit and scope of the present invention.

At step 5903, the system “looks” for features that are visible withinthe photo container boundaries and that might be central to thatparticular photo. For example, if the photo is of a person, the systemmay focus on the face of the person as the most important feature ofthat photo to use in a mosaic pattern. If the photo is of a church,perhaps the system will focus on the entry way of the church or thesignage or name placard of the church. If the photo is of an animal, thesystem may attempt to focus on the entire animal or a part of theanimal.

At step 5904, the system determines if a distinguishing feature of ananalyzed photo has been identified. In one embodiment, the system islimited in that it may only focus on features of a photo that arevisible (user's perspective) within a given photo slot. If at step 5904,the system does not identify a specific feature in a photo analyzed atstep 5903, the process may move to step 5908 where the system may get anext photo and skip or “forget” the photo for which a feature was notidentified. It is noted herein that a mosaic pattern might be created ofa few photos in a project, several photos in a project, or all of thephotos in a project.

The number of photos used in a mosaic design depends on the pool ofavailable photos in the project that have identifiable features. In oneembodiment, a photo that has no system-identifiable feature might stillbe used in a mosaic design without departing from the spirit and scopeof the present invention. Identification of primary features in photosis practiced for aesthetic purposes. Pattern recognition of shapes canbe used as one method for optically identifying the image featuressupported generically in a database. For example, a pattern recognitiondatabase can be used to store shapes for comparison in photo analysis.

If the system identifies a primary feature in a photo at step 5904, theprocess moves to step 5905 where the system centers the featureaccording to the mosaic tile footprint of the tile of the mosaic patternthat will present the feature of that particular photo. In oneembodiment, the mosaic pattern of tiles represents the compilation ofthe selected photos centered in the photo slots or photo wells used inthe project. In this case, a tile is a photo container with the photoinside. The photo-containers may be presented at specific core aspectratios such as portrait, landscape, or square. A combination of theseaspect ratios might be incorporated into a mosaic design.

In another embodiment, the photos selected for the mosaic design arecentered in mosaic tiles that function as photo containers but are notthe same photo slots used in the project. The tiles capture the desiredfeatures identified in the photos and that are centered with respect tothe tile boundaries at step 5905. In one embodiment the photos arecropped to fit the respective tile footprints at step 5906. In anotherembodiment, the photo portion extending past the tile boundaries ismasked and is transparent to the user.

At step 5907, the system determines if it is finished applying the photofeatures to the mosaic tiles. If the system is not finished at step5907, the process resolves back to step 5908 where the system gets thenext photo for processing. If at step 5907, the system is finished withall the design tiles, the system assembles a mosaic pattern using theprocessed “photo tiles” at step 5909. In one embodiment, the system mayhave several pre-conceived design models for mosaic patterns and mayselect one of those designs based on the number of photos successfullyprocessed in the project. A pattern may contain only a few photos or itmay contain many photos without departing from the spirit and scope ofthe present invention.

In one embodiment, the system may use theme-based images that are systemsourced and therefore not photos submitted by a user in combination withphotos submitted by the user in a same mosaic pattern. Motivation forusing filler images might be that a given mosaic pattern has X number oftiles but the user only has Y number of photos in the project where Y is<X. It is noted herein that the die-cutting service vendor, whether itis the print vendor or not must have the applicable dies for cutting themosaic pattern through the book cover.

At step 5910, the system positions the mosaic pattern on a canvas page(digital) dedicated for the purpose facing the inside cover of the book.If the pattern is large it may be centered on the cover. If the patternis small it may be moved to an off-center coordinate. In one embodimentmore than one mosaic pattern may be generated for a same project. Itwill be apparent to one with skill in the art of die-cutting that themethod of the invention can be practiced with image-based products otherthan photo books, such as photo cards, photo posters, photo calendars,etc. without departing from the spirit and scope of the presentinvention. It will also be apparent that the pattern may be presented onanother page in a product other than the front cover such as an internalpage or the back cover, for example.

Mobile Application

Referring now to FIG. 1, the present invention may be practiced on amobile appliance such as on laptop computing appliance 122 or on smartphone 124. Bandwidth dependant devices may be optimized to betterpractice the present invention, in one embodiment, through provision ofa mobile application that is tailored for such mobile appliances.

FIG. 60 is an architectural overview of a fulfillment network 6000supporting the initiation, building, and editing of image and ortext-based projects from a wireless computing appliance. Network 6000,which may be the Internet in one embodiment, supports a Web server (WS)6002 hosting a Website 6006 where users such as one operating a wirelesscomputing appliance 6005 may register with a service to initiate, build,edit, and eventually purchase image and text-based projects like photobooks, calendars, cards, posters, and other products. WS 6002 hasconnection to a data repository 6003 adapted to contain customer data ofcustomers of the service.

Customer data may include customer identification such as name, address,and contact information. Customer data may also include log-in data,billing information, and transaction history. In one embodiment, when auser registers for the service at Website 6006, that user may registerto access the service from a wireless appliance like wireless computingappliance 6005. Wireless computing appliance 6005 may be a smart phone,an I-pad, notebook, nook, or android device with a display and inputmechanism such as a physical “dockable” keyboard or touch screen.

In this embodiment, a user operating appliance 6005 connects to Website6006 hosted on server 6002 through a wireless carrier network (WCN) 6001and a wireless Internet service provider (WISP) 6014. Wireless mobileappliances may experience some bandwidth limitations from time to timethat are inherent with many wireless methods of connecting to theInternet. Mobile computing appliances are many-times referred to as“light” devices, which may rely on “light” operating systems, wirelessapplication protocol (WAP) Internet page service protocols, and so on tohelp optimize Internet services made available through such appliances.During the registration process at Website 6006, the user may requestand register for a special “light” service for wireless appliances suchas appliance 6005.

Network 6000 supports a runtime server (RS) 6004 that facilitatesproject initiation, build, edit, preview, and fulfillment. Server 6004hosts software (SW) 6007 that includes all of the functional softwarecomponents for facilitating the service for both wired and wirelesslyconnected computing appliances. Computing appliance 6005 includes adisplay 6012. In this embodiment, display 6012 may be a touch screendisplay. In this embodiment, display 6012 includes an open image- andtext-based project 6013. Project 6013 may be a photo book or a similartype of image and or text-based project.

The user operating appliance 6005 logs into Web site 6006 and isredirected to RS 6004 to engage in project services. In this regard, Website 6006 may offer a mobile application to the user, which is operableon a lightweight operating system (OS) and browser version. In oneembodiment the user downloads the mobile application indicated in thisexample as software (SW) 6011 executing from computing appliance 6005.SW 6011 includes at least a user interface (UI) for enabling at leastpreview of project 6013. The UI may include some build tools such as aphoto selector component similar to photo browser interface 5807 and aphoto management tray like photo tray 5808 described above withreference to the description of FIG. 58.

The browser interface for selecting photos (5807) enables the useroperating appliance 6005 to select their photos from a device library ofphotos contained in internal or peripheral memory. The image or photomanagement tray (5808) enables the user to organize and preview theselected photos including visualizing the correct aspect ratios andproperties of the selected photos and visualizing the order that theselected photos will appear in project 6013. The UI may also includelinks to server-hosted editing tools for font, graphics, etc.

SW 6011 may also include one or more components to support purchasingtransactions and order tracking of purchased projects. Such a componentor components may include an interface for entering shipping informationand payment information and an update interface for tracking the orderrelative to production and shipping of the order. In one embodiment, themobile application is first downloaded from Web site 6006 and installedon computing appliance as SW 6011. In another embodiment, the userregisters to receive the application download, but the mobileapplication is actually downloaded from server 6004 when the user firstengages the server to initiate a project. In one embodiment, the mobileapplication includes identification of a proxy server and a bandwidthoptimizer for compressing images and graphics for faster download to andupload from mobile computing appliance 6005.

In use of the present invention, a user operating appliance 6005connects to Web site 6006 and registers for services including serviceto a wireless mobile appliance such as a smart phone, for example. Atproject initiation, server 6002 or server 6004 downloads the mobileapplication (SW 6011) to mobile computing appliance 6005. RS 6004 hasconnection to data repositories 6008 containing GUI components includingcomponents for wireless applications, 6009 containing project graphics,and 6010 containing project data such as text and templates, etc.

Once SW 6011 is installed on appliance 6005, the user may re-connect toRS 6004 and initiate a project for build. Project 6013 is in display onappliance 6005. The user may build edit, order, preview, and track theprogress of the order. Wireless application 6011 may also benefit fromvarious bandwidth saving optimizations described previously in thisspecification like processes revealed in the flow charts of FIG. 49,FIG. 50, and FIG. 55 of this specification relative to image processingand local caching of assets on the client device.

Text Input on Space-Limited Displays

Space limited displays, such as those on mobile phones for example, arenot large enough to facilitate form filling on multi-line forms. Theinventors provide a method for populating multi-line forms on a devicethat has a space-constrained display.

FIG. 61 is a process flow chart 6100 depicting steps for form fieldpopulation on a space-constrained device according to an embodiment ofthe present invention. With respect to chart 6100, it is assumed thatthe user has logged into the service and has initiated a process thatinvolves form filling. It is noted herein that digital forms might berequired to be populated for processes like user registration forservices and, in certain circumstances, when a user is involved inproject creation. A digital form is an electronic form or text interfacethat includes more than one form field for data population.

At step 6102, a user requests a digital form for population. Such arequest may be a direct request where the form is specified or a generalrequest for initiation of a process where populating a digital form isrequired for process initiation or in process resolution. At step 6103,the system makes a determination whether the user is operating anappliance with limited display space such as a cell phone, smart phone.If the system determines that the user is not operating a display orspace limited appliance, the system serves the regular complete digitalform for data population at step 6104. In this case, the user populatesthe form at step 6105 and submits the populated form at step 6106. Theprocess for the user then ends at step 6115.

If the system determines that the user is operating a space-limitedappliance at step 6103, the system may serve an individual form field ofthe digital form at step 6107. The individual form field represents adigital form field that is modular and that can be docked to andundocked from the parent form. Typically, the first form field served tothe user is the first form field appearing in the digital form thatrequires some data population. In one embodiment, the parent digitalform may be viewed as an object container and the individual form fieldsare objects that can be transferred over the network as single objectstracked to the parent object or object container such as by tagging orother indicative marking methods.

At step 6108, the user receives and populates the individual form fieldusing a physical or touch screen enabled key pad, key board. In oneembodiment, the individual form field may be auto populated using dataalready known to the system when served to the user at step 6107. Inthis case, the user may simply preview the individual form field andmake any manual data corrections that may be necessary at step 6108. Atstep 6109, the system determines if the individual form field has beenpopulated. The system may determine this in one embodiment by monitoringthe process on the user's appliance through the server/clientconnection.

If the system determines that the individual form field is not populatedat step 6108, the process experiences a delay at step 6110 before theprocess resolves back to step 6109 where the system again attempts tomake the determination. If the system determines that the individualform field is populated at step 6109, the system checks at step 6111whether all of the individual form fields have been populated withrespect to the parent digital form. In one embodiment, the user invokesa command such as “next” or “enter” when the user has finished enteringdata into an individual form field telling the system to serve ordisplay the next individual form field for data population. In oneembodiment, the system simply accounts for any un-served form fields ofthe digital form waiting for service to the user appliance. In anotherembodiment, the system might have served all of the individual formfields of the digital form to the user at step 6107 but displays onlyone form field at a time to the user, such as from the user's localcache memory.

If the system determines that all of the form fields associated with thedigital form requested at step 6102 have not been populated at step6111, the system may serve the next unpopulated individual form field atstep 6112. In a case where all of the form fields are in user cache, thesystem may simply display the next unpopulated form field for datapopulation. In one embodiment, once a form field is populated, thesystem displays the next form field over the last populated form field.In another embodiment, the previous form field may be minimized or movedout of the display space to make room for the next form field. In apreferred embodiment, the individual form fields contain “helper text”that is instructive of the type and form of data required in the formfield such as a label or an example text. Such helper text disappearswhen the user begins populating the form field.

The process then loops back to step 6108 where the user fills out theindividual form field displayed. The process then repeats steps 6109 and6111 until the system makes a determination that all of the individualform fields served in steps 6107 and 6112 have been populated. In thiscase, the process moves to step 6113 where all of the populated formfields may be collected from the user appliance (uploaded to theserver). In one embodiment, the system may collect populated form fieldsserially as they are populated using an automated upload feature thatautomatically uploads finished form fields.

At step 6114, the system reconstructs the parent digital form containingall of the populated form fields in their correct docking points in theparent form. The system may, in one embodiment, reconstruct the digitalform on the client appliance before uploading the entire form. Inanother embodiment, the system may collect all of the populated formfields in one data upload session and reconstruct the parent form at theserver. The process then ends at step 6115. In one embodiment, when theuser is populating the last unpopulated form field, a system pop upnotification may appear informing the user that there are no more fieldsto populate with a submission button for the user to initiate upload ofall the populated form fields or form if it is reconstructed on theclient appliance.

Pre-Filling Address Based on Zip Code Entry

Form population on a mobile device is more time consuming and tediousthan on a desktop or notebook computing appliance as was describedfurther above. The inventors provide a method for auto population ofcertain address information for clients whom are required to submit suchinformation relative to form processing.

FIG. 62 is a process flow chart 6200 depicting steps for auto-populatingaddress information for clients filling out forms according to anembodiment of the present invention. At step 6201, a user connects tothe service. Typically, the user will connect to a web site to accessthe service. At step 6202, the service downloads a United States zipcode database onto the user's connected computing appliance. Thedatabase includes the zip code and correlating city, county, and state.

At step 6203, the user requests a form. The request may be tied toinitiation of a process like user registration, a transaction, or someother process requiring form filling as part of the interaction. At step6204, the form is displayed on the user's appliance display screen. Theform may be server-hosted or downloaded to the appliance and displayed.At step 6205, the user populates the form with the required data. Duringthis process, the system monitors the activity of the user and thereforemay detect when the user enters a zip code onto the form. In a preferredembodiment, the form field for containing the zip code is presented inthe form before fields requiring other address information.

At step 6206, the system determines if the user has entered a zip code.If the system determines at step 6206 that a zip code has not yet beenentered onto the form, the process moves to step 6207 for a short delayor pause and resolves back to the determination step 6206. If the systemdetermines that a zip code was entered onto the form, the system usesthe zip code to perform a lookup on the user appliance to get theassociated city and state, and if required the associated county at step6208.

At step 6209, the system auto-populates the appropriate form fields withthe city, state, and county (if required). At step 6210, the systemdetermines if the form has been completed. If at step 6210, the systemdetermines that the form is not complete, the process resolves back tostep 6207 for a short delay or pause, and then back to step 6206 wherethe system continues to look for a next zip code entry. In oneembodiment, the system is aware how many fields on the form are adaptedto accept a zip code. If the nature of the form is such that there isonly one field for a zip code, the process ends before step 6210. If thesystem determines that the form is complete at step 6210, the processends at step 6211.

Referring to steps 6206 through 6210, if a form is presented that mayrequire multiple zip code entries like a shipping form documentingmultiple different addresses, for example, the process continues to loopthrough until the system determines the form is complete. In oneembodiment, zip codes are correlated to area codes as well as to city,county, and state. In this case, any telephone field may beautomatically populated with the correct area code. In a situation wherea zip code area includes more than one area code, the system may waitfor the user to input a street name into an address field. The systemmay then use this data to obtain the correct area code for the user'stelephone number.

It is noted herein that in one embodiment the process works whether ornot the user is connected to a server while filling out the form. Inthis case, the zip code database is stored on the user's appliance, anda downloaded system component functions independently from the onlineserver to perform the data lookups on the user's system and autopopulates the form with the returned data. In another embodiment, theuser must be connected to the server for the system component to monitorthe process of filling in the form and to retrieve data from the user'sappliance for auto-population into the form fields.

In an embodiment where there is more than one field on the form thatrequires the same zip code relative to the user's location, the systemautomatically populates the associated city and state data for all ofthe locations or fields on the form that require the information basedon the single lookup using the user-entered zip code. If the userchanges a zip code in the field, the system detects the change and getsthe associated city and state data overwriting previous data ifrequired. In one embodiment, the system may use global positioningsatellite (GPS) data if available to inform the user of the correct zipcode before the form is filled. In this case, if the user confirms thesystem finding of the correct zip code, the system will perform thelookup; get the associated city and state information; and auto-populatethe form ahead of the user interacting with the form.

Order Preview on Mobile Appliance

In many instances, users who conduct transactions receive some textualindication of the transaction order such as an order number,confirmation number, or some other text data that may provide little orno contextual information relative to the transaction and the associatedinformation like shipping data, subject matter, content, etc. Therefore,the inventors provide order confirmation indicia that a user mayinteract with, through touch screen interface, to preview thetransaction and associated data.

FIG. 63 is a block diagram illustrating a mobile appliance 6300supporting preview of an order for an image and or text-based productcreated from a mobile application according to an embodiment of thepresent invention. Mobile appliance 6300 may be an I-pad device, anandroid device, a notebook, or a smart phone having a touch screeninterface. Mobile appliance 6300 includes a touch screen-enabled display6301. Mobile appliance 6300 may include a close proximity wirelessinterface technology for communicating with a local transaction serveror a transaction machine interface in order to synchronize with theruntime server to obtain relevant order information relative to ordersplaced for image and or text-based products created using appliance 6300or created using another computing appliance such as a desktop or laptopcomputing appliance.

In this embodiment, the transaction service provides visual indicia 6302(1-n) in the form of interactive icons that provide visual indication ofproducts ordered by the user in the past. Indicia 6302 (1-n) providevisual indication to the user of which products were ordered. In thisembodiment, the correct aspect ratio of the associated product isvisually represented by each icon. In one embodiment, the icons arearranged in a chronological order revealing an order history startingwith the most recent order reading from the left to the right. Indicia6302 (1-n) may be downloaded from a transaction server during atransaction process, or obtained through a wireless session operationwith any enabled transaction terminal having access to the data.

During the order process, the service may capture the likeness of theimage- and or text-based project ordered such as core aspect ratio, andan interactive thumbnail view of the front cover of the ordered project.A user may select and interact with an icon 6302 (1-n) using touchscreen input to preview at least order and shipping information for anyproduct ordered. For example, touching one of icons 6302 (1-n) may causean information window 6303 to appear in an unused portion of display6301. Information window 6303 provides at least a title, order date,order confirmation order, estimated shipping date, total price paid bythe user, and indication of any discounts applied to the purchase.

Other information may also be included in information window 6303without departing from the spirit and scope of the invention such asshipping address, identification of drop ship recipients and theiraddresses, and information about what other products are typicallyordered by users that ordered this product. In one embodiment, a usermay further interact with an icon 6302 (1-n), such as by double tappingthe icon to cause a preview interface 6304 to appear in touch screeninterface 6301. In one embodiment, preview interface 6304 provides aread-only preview of the ordered product at a reduced size andresolution for convenient preview of the project on the mobileappliance.

Preview interface 6304 enables at least limited (read only) navigationof and digital expansion of one or more ordered projects indicated indisplay 6301 as indicia 6302 (1-n). In one embodiment, a user mayfurther interact with preview interface 6304 to navigate to the serverand access the actual server-hasted version of the project for continuedediting of the project in session. In this case, the preview maytransform to a fully editable display of the project expanded in displayto utilize the full footprint of display 6301.

Facilitating Photo Contributions

FIG. 64 is an architectural overview of a fulfillment network 6400supporting photo contribution to image-based projects created through agraphics user interface. Network 6400 includes the Internet networkdepicted herein by a network backbone 6401. Network backbone 6401represents all of the lines, equipment and access points that make upthe Internet network as a whole, including any connected sub-networks.Therefore, there are no geographic limitations to the practice of thepresent invention.

Backbone 6401 supports a Web server (WS) 6403. Web server 6403 hosts aWeb site 6407. Web site 6407 serves as a client access point to theservice of the invention. Web server 6403 has connection to a datarepository 6405 adapted to contain client data. Client data 6405 maycontain information about clients of the service including, but notlimited to, identification, billing data, account information, purchaserecords, security information, log-in information, project information,and any other data deemed important to record for clients of theservice.

Backbone 6401 supports a runtime server (RS) 6404. Server 6404 includesat least one processor and a data repository and software (SW) 6411running from a non-transitory medium on the server. Server 6404 includesa graphics user interface (GUI) 6408 adapted to enable project creationin a server-hosted embodiment. RS 6404 has connection to a datarepository 6406 adapted to contain photos that are uploaded from usersand friends and family of those users.

In this example, a user operating a computing appliance 6412 hasconnection to RS 6404 through access network 6402, a data switch orbridge 6410, and backbone 6401. The user operating appliance 6412 has aproject being created through the GUI displayed on appliance 6412. Intypical fashion, the user connects with Web site 6407 running on sever6403 and logs in to see their projects. When a project is executed, aserver redirect connects the user with RS 6404 hosting the GUI used tobuild and display projects.

A user that is building or editing a project hosted on server 6404 maynot want to collaborate with other users. However, the user may desireto incorporate photos from one or more family and or friends whom areknown to the user, but whom may not be active collaborators and mightnot have an account registered at the service. Such friends and orfamily members of the user operating computing appliance 6412 aredepicted herein as computing appliances 6409 (1-n).

When the user operating appliance 6412 is in session with RS 6404 andbuilding a project, the system using a monitoring component that detectsthe operational state of the user, may prompt the user to determine ifthe user wants to get friends and family members to contribute photosthe project the user has open. If the user agrees, SW 6411 generates atleast a digital notification or message that can be delivered to users6409 (1-n), for example via email address cell phone number (SMS, MMS),with a browser link to server 6404. The notification or message maycontain a description of the project and an invitation to the user tocontribute one or more photos to the project. The notification mayinclude system requirements for photo dimensions, pixel resolution, andcontent. In one embodiment, the user may re-distribute the software tothe other users instead of the system.

SW 6411 may include a light-weight component distributed as SW onappliances 6409 (1-n) that might function as a browser (BSR) plug-in orextension that provides a photo uploading interface for the contributorsto use to gather and vet the photos. As photos are contributed to aproject, the project owner may be notified that there are new photosavailable from friends or family members. In one embodiment, SW 6411 mayalso cause distribution of a lightweight order interface to thepotential contributors along with an interface for directly uploadingphotos to server 6404.

The order interface may allow the contributors to directly purchase acompleted project that they contributed photos to. In one embodiment,the interfaces may also provide incentive to contributors to registerwith the service to create their own projects. Potential contributorsmay also re-distribute the interface to other friends and family. Thephoto contribution interface and order interface may be distributed toany online social interaction accounts owned by friends or familymembers instead of sending them directly to their end devices. In thiscase, the interface(s) may be downloaded and installed for use in photocontribution and product ordering.

In one embodiment, when the original user opens a project, they maybrowse through photos that were contributed to a pool of availablephotos and metadata about the photo contributions may include thecontributor identifications. Photos uploaded to the server bycontributors may be stored in repository 6406 for use by projectbuilders.

Colorable Assets

FIG. 65 is an architectural overview of a fulfillment network 6500supporting provision of colorable assets to users engaged in buildingand editing image-based projects through a graphics user interface(GUI). Network 6500, which may be the Internet in one embodiment, whichis further illustrated by a network backbone 6501. Network backbone 6501represents all of the lines, equipment, and access points that make upthe Internet network as a whole, including any connected sub-networks.Therefore, there are no geographic limitations to the practice of theinvention.

Network backbone 6501 supports a Web server (WS) 6503 hosting a Website6507 where users such as one operating a wireless computing appliance6511 may register with a service to initiate, build, edit, andeventually purchase image and text-based projects like photo books,calendars, cards, posters, and other products. WS 6507 has connection toa data repository 6505 adapted to contain customer data of customers ofthe service.

Customer data may include customer identification such as name, address,and contact information. Customer data may also include log-in data,billing information, and transaction history. Wireless computingappliance 6511 may be a smart phone, an I-pad, notebook, nook, orandroid device with a display and input mechanism such as a physical“dockable” keyboard or touch screen.

In this embodiment, a user operating appliance 6511 connects to Website6507 hosted on server 6503 through a wireless carrier network (WCN) 6502and a wireless Internet service provider (WISP) 6514. Appliance 6511 maybe a desktop computing appliance, or a laptop computing applianceconnected to the network through a wired connection without departingfrom the spirit and scope of the present invention.

Network backbone 6501 supports a runtime server (RS) 6504 thatfacilitates project initiation, build, edit, preview, and fulfillment.Server 6504 hosts software (SW) 6506 that includes all of the functionalsoftware components for facilitating the service for both wired andwirelessly connected computing appliances. Computing appliance 6511includes a display 6512. In this embodiment, display 6512 may be a touchscreen display. In this embodiment, display 6512 includes an open image-and text-based project 6513. Project 6513 may be a photo book or asimilar type of image and or text-based project.

RS 6505 has connection to a data repository 6510 containing project dataabout user projects registered with the system. RS 6504 has connectionto a data repository containing project photos provided by users forinclusion into their projects. RS 56504 has connection to a datarepository 6508 containing theme-based assets like stickers, shapes,backgrounds, and general clipart assets that are system designed assetsfor inclusion into projects. The term colorable assets refers to systemdesigned graphics that contain regions that are color neutral whereby auser may choose which color to fill into the color neutral regions ofthe asset.

Project 6513 includes photos, text, and system designed assets referredto as stickers or shapes that are available to users for decoratingproject pages along with user-supplied photos. System designed assetsmay also include borders, backgrounds, photo slots, and any other systemdesigned graphics that are made available to users through the system.

In this example, a sticker 6515 in the shape of a star is illustrated inan expanded view from project 6513. Sticker 6515 contains a colorableregion 6516 consisting of a five-sided region central to the star shapeand geometrically aligned with the five star points. Region 6516 may bedefined and assigned to asset 6515 as a color neutral region that a usermay color in by selecting a color and filling the region with thatcolor. Region 6516 may be designed according to any geometric patternand such design incorporates the shape and dimensions of the asset towhich it is assigned. In one embodiment, a substantially large colorneutral region is assigned to or attributed to an asset while a verysmall area of the asset may be reserved for default coloring.

There may be more than one colorable region attributed to asset 6515without departing from the spirit and scope of the present invention.Likewise, there may be more than one default colored region representingthe area of the asset that does not include the color neutral region orregions. Colorable regions and default colored regions may coexist onthe asset in a symmetrical relationship. For example, the five starpoints of assert 6515 may be colored by default while the center regionis color neutral and where the center color neutral region issymmetrically complimented by the default colored regions. Coloring thecentral region may be accomplished using a color picker and selecting acolor to use. The region may then be selected by the user and a filloperation may be used to color the region with the selected color. In amore complicated example, a background graphic of grass, sky, andflowers may have colorable regions representing the flower petals. Inthis case the user may select the flowers and color the flower petalswith a color chosen by the user.

In one embodiment of the present invention, there may be more than onecolorable region provided on an asset where the user may be enabled tocolor those regions with different colors, for example, one region red,the adjacent region blue, etc. In this way, assets are not predisposedto limited application because of a default color that cannot be changedby the user. In one embodiment, a user is prompted if they want to haveaccess to colorable assets for their project. If they agree, theseassets might be served to the user for consideration. In one embodiment,the user selects a mode that tells the server to present colorableassets to the user because the user has selected a manual coloring modeduring the project building or editing process. A color neutral regionof an asset might be filled with a single color, a blended color, or arange of colors that may be added according to certain effects likecolor separation or saturation effects. To illustrate, consider region6516 colored red at one end where the color red slowly changes to blueat the far end with the gradient shades between red and blue disposed inbetween.

In one embodiment, a user may color a color neutral region by hand usinga color picker and scribble tool or writing tool that allows the user towrite graffiti or other hand written shading or marking using one ormore than one color. In one embodiment, the user may be enabled toswitch color designation between colorable regions and default coloredregions of an asset by selecting a tool to invert the color assignmentsafter the colorable region is filled in. In one embodiment,color-customized assets are retained by the system and may be reused byother users. In one embodiment, a user working on a canvas containingmultiple colorable assets might color the asset regions and save asnapshot of the result to use in comparison to other colors that mightbe selected or suggested by the system based on a color matchingalgorithm. There are many possibilities.

It will be apparent to one with skill in the art that the system of theinvention may be provided using some or all of the mentioned featuresand components without departing from the spirit and scope of thepresent invention. It will also be apparent to the skilled artisan thatthe embodiments described above are specific examples of a singlebroader invention which may have greater scope than any of the singulardescriptions taught. There may be many alterations made in thedescriptions without departing from the spirit and scope of the presentinvention.

What is claimed is:
 1. A system for creating and editing image-basedprojects comprising: a server connected to a network, the server havingaccess to least one processor and a data repository, the serverincluding a non-transitory physical medium; and software running fromthe non-transitory physical medium, the software providing: a firstfunction for establishing a client server connection between the serverand at least one user-operated computing appliance connected to thenetwork; a second function for initiating and maintaining an active datasession between one or more users involved in project creation and or inproject editing through a graphics user interface (GUI); a thirdfunction for analyzing in at least three dimensional values, all of thecolors of an asset presented in the image-based project; a fourthfunction for creating a histogram of the asset based on the analysis bythe third function; a fifth function for blending the histogram byadding new color contributions to augment the original colors inrelevancy; and a sixth function for storing the blended histogram withthe asset.
 2. The system of claim 1, wherein the network is the Internetnetwork.
 3. The system of claim 1, wherein a color palette is definedfrom the original analysis of the three dimensional color values.
 4. Thesystem of claim 1, wherein tag terms and color terms are used to searchfor assets that have been analyzed and stored for use.
 5. The system ofclaim 3, wherein contributions added to the color palette for an assetare added as a result of a color blending operation.